22

This is to share a case study about the "fastest gun in the west" issue discussed on Academia.SE here and more broadly here.

On this question, the sequence of events was:

  • Buffy, Guest, and I all submitted answers on literally the same minute.
  • Buffy's answer was immediately upvoted, and from there people piled on. Within a ~day, he was ahead by 40 votes; I think the score was something like 100-60. Our answers are very similar, so it's a bit strange that there was such a difference (I blame the dog picture...).
  • Then, OP accepted my answer, meaning that my answer is displayed first, despite having fewer votes. The gap quickly began to shrink, and a few days later, I am ahead by 10, 216-205.

Since neither of us over edited our answers, this is a clear illustration that the top-rated answer gets a huge bias. [For that matter, I've seen similar behavior in close voting -- hard to stop the close train once it gets rolling -- but that's a separate topic.]

Of course, I don't particularly care about my imaginary points, but it does seem like an obvious weakness:

  • Better answers submitted after ~10 votes are in are unlikely to be read
  • This affects new users who can't even comment yet
  • As Buffy points out, the same is true for downvotes too (though it seems like we downvote less on this SE than others), and arbitrary downvotes are a sure way to discourage new users.

We've discussed this before, but I guess my questions are:

  • What could we do, mechanically? Does SE support solutions like hiding the vote totals for the first few hours, or is there simply nothing that can be done without getting the powers that be to write entirely new code?
  • If we do have the power to make such changes, has the community already decided not to? The questions I linked seemed open to such changes, but nothing happened.
  • For this particular case, you can blame voters from Hot Network Questions ;) . Anyway, what is the purpose of hiding the vote totals for the first few hours? Do you mean that voters are biased to already highly-upvoted answers? Does that also include hiding it from the OP? Will that also affect the answer order, since one of the sort is by "votes"? – Andrew T. Jan 29 at 8:19
  • 2
    The point is: does it really matter? – Massimo Ortolano Jan 29 at 9:11
  • 8
    Actually, "piling on" by voters also happens in the negative direction. I think that the more likely reason for this behavior is the inclusion of some simple phrase within the answer that people either especially like or dislike for some reason. I've seen both quick up and down voting on some of my recent answers. Some of them are hard to explain. – Buffy Jan 29 at 12:20
  • 3
    @MassimoOrtolano - I would advocate the perspective that it does, particularly for newbies on SE, as many/most SE privileges are rep based. Post 3k rep points, perhaps not so much. The psychological biases that people carry in voting (this post being an example) can potentially become a barrier to contributions. e.g. X saw a question that he could contribute to, something beyond the existing 5 answers, with the top answer at +40. We have a problem if X thinks "Why bother posting an answer, no one will bother reading past the top 2 answers. It will collect say 2-3 votes and will stay buried". – 299792458 Jan 29 at 15:36
  • 1
    @MassimoOrtolano (contd.) While SE has inserted badges to denounce this reasoning, we perhaps can't deny that it does indeed happen. Even more so on the relatively softer Academia.SE, as opposed to other more technical stacks, where the validity of an answer is more objective (yes or no). In my early days on SE, I personally found this to be some sort of rich-gets-richer system, and it was pretty discouraging. What this post does is, it very interestingly brings out is a case study perspective on this issue, which totally supports the observation. I think it warrants a discussion. – 299792458 Jan 29 at 15:45
  • Andrew - Yes, I think it's clear voters are biased to upvoted answers, and hiding vote totals for the first few hours would force people to think critically before voting. This may introduce additional complications, so I'm not necessarily advocating that particular solution just yet, but offhand it seems like a good idea. Massimo - I think it does matter, though admittedly more for new users and downvotes than for upvotes on high-rep users, Buffy - agree, updated my post, The_Dark_Side - agree, well said, thank you. – cag51 Jan 29 at 15:51
  • I think hiding vote totals for a bit isn't a bad idea, but it has the downside of someone finding the question and not knowing what the community thinks the best answer is for a while. Maybe not important on this site as much, but I could see SE being reluctant to implement it on a technical site, lest someone follow really bad advice that was actually downvoted to hell but no one could see – Azor Ahai Jan 29 at 22:14
  • 1
    @TheDarkSide, for the record, I will seldom post to a question with five answers if I think that, collectively, they provide a sufficient answer to the OP. For me, having a big rep only means that I'm being more useful to people than otherwise. But it also gives me the courage to occasionally say things that I'm pretty sure will be controversial. Not everyone wants to read some things that they need to read - especially on ethical issues (and copyright). – Buffy Jan 30 at 13:12
  • This probably belongs more on the main meta, though it's been discussed there already as well. – Bryan Krause Jan 30 at 18:10
  • 2
    Here is an example of negative piling on: academia.stackexchange.com/a/124038/75368. I say some things that (IMO) are a bit misinterpreted, but that people really don't want to hear, whether accurate or not. – Buffy Jan 30 at 19:08
  • @Buffy - Haha, I was thinking the same thing when I saw that thread :-) Though in that case, I was rather hoping to see some insight about why this would or would not be illegal/unethical/messy, rather than just advising to take the safe course and avoid risking any blowback (which is what I would do, since I don't want to find out "what could go wrong" the hard way) – cag51 Jan 30 at 20:45
  • 4
    If you attempt to solve this by hiding vote totals, you also need to randomize the order of posts seen by different viewers. Otherwise, a reasonably good answer at the top of the list will also collect "that's OK, upvote it, move on to the next thread" bias. The basic issue, IMO, is that people in general like expressing their opinion (i.e. voting) more than they like mental effort (i.e. choosing carefully what to vote for). – alephzero Feb 9 at 17:24
  • I think the problem might come because people only vote on the posts they read. Top posts get read, bottom posts get neglected. Especially on Academia where the answers tend to be lengthy. – LN6595 Feb 11 at 3:19
9

It's not something we can solve as academia.SE.

As you noted, it's been a known thing for many years over at meta.stackexchange.com (Fastest Gun in the West Problem). They even have a likely solution noted there, which is what Reddit adopted some years back. I imagine there must be counterarguments as to why that isn't a great idea. But regardless, so far as I know it isn't something that Stackexchange is planning to deal with :-(

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .