-10

Probably half of the question have some sort of bickering on what's appropriate. There are questions as open ended as "What should I do with my academic career" being up voted, when there's an infinite number of potential answers, and no one can guarantee that the response is going to be helpful. Then there are other similar questions being down voted as too broad.

For example Should I prefer PhD programs with "higher quality" students? Is a shopping question on what phd program to select. +4 up votes.

Right below it https://academia.stackexchange.com/questions/86669/phd-is-75-done-in-russia-any-options-in-europe-uk-citizen Another shopping question on what phd program to select. -4 down votes.

There are social etiquette questions about how to answer emails that have almost nothing to do with academia being highly prioritized. How to deal with an inappropriate greeting in an email?

And another question about how to respond to emails dealing with research paper submissions being down voted and ignored. What to do when you have not received a response three weeks after submitting minor revisions?

If the purpose is to form a consistently high standard, how are you ensuring that you're meeting that goal? Because that goal appears to be different depending on who's viewing your question.

If the purpose is to exclude rude and offensive behavior and to create a welcoming environment, I don't really see that either. Obvious insults may be quickly deleted but the questions are frequently met with inconsistently enforced rules, dismissive links to solutions that only tangentially answer the questions, or just snark. And a simpler rule set "No cursing, no racism, sexism, etc." could just as easily create that sort of environment.

This question is about as basic meta as it gets. How to check internal measurements of standards. Every serious company has constant checks on their standards. Hospitals aren't willy nilly winging their prescriptions for instance. And everyone is probably willing to admit more precision is better than less, and more solutions are more valuable than few solutions. No one wants enough variability in their anesthesia to kill them. Democracies where voters get to decide value are never perfect. Elevating individuals to judge others always leads to flawed decision making. This is a simple question "How are you measuring whether your judges are doing their job?" or "How are you ensuring that your democracy is meeting the needs of the community?" that every organization has to answer. The most meta of questions to be asked. And from what I've seen I expect it to be down voted.

  • I'd prefered if you'd written "How are we measuring whether our judges are doing their job?" and "How are we ensuring that our democracy is meeting the needs of the community?" – Dirk Mar 29 '17 at 9:35
4

Your examples

For example Should I prefer PhD programs with "higher quality" students? Is a shopping question on what phd program to select. +4 up votes.

Right below it https://academia.stackexchange.com/questions/86669/phd-is-75-done-in-russia-any-options-in-europe-uk-citizen Another shopping question on what phd program to select. -4 down votes.

There is a crucial difference between those questions. The first one is asking whether a certain aspect is a relevant criterion to select a program; the second one is asking for a specific individual program. Or with other words: The first question is asking how to shop; the second question is asking what to shop. The latter what we call a shopping question. I tried to make a FAQ on this, explaining what makes a shopping question and why they are bad, but it did not receive much attention yet.

There are social etiquette questions about how to answer emails that have almost nothing to do with academia being highly prioritized. How to deal with an inappropriate greeting in an email?

I disagree that this has nothing to do with academia. Sure, such a situation could occur elsewhere, but then you would evaluate it differently (academia’s e-mail etiquette differs from business e-mail etiquette) and the ways to react to it are different: If a representative of a company was rude to me, I can report this to their superior; in a student–teacher relationship, this is not possible.

Note that this question got an insane amount of attention and hence votes due to being a hot network question.

And another question about how to respond to emails dealing with research paper submissions being down voted and ignored. What to do when you have not received a response three weeks after submitting minor revisions?

This question is hardly comparable to the previous one, because it is about an entirely different situation. The problem with this kind of question is that we have dozens of it and there is little to answer for us. In fact, this lead me to propose to make a canonical question on this.

Your general question

If the purpose is to form a consistently high standard, how are you ensuring that you're meeting that goal? Because that goal appears to be different depending on who's viewing your question.

That is indeed the goal. We are ensuring to meet that goal by requiring five close votes on a question, allowing users¹ to vote on posts, and allowing users¹ to participate in improving questions. Also there are mechanisms to reöpen questions.

Of course, this system is not perfect, but it is a viable compromise between fairness, a working community, and effort.

How are you measuring whether your judges are doing their job?

If anybody¹ has the feeling that our closing behaviour needs changing or a specific question was wrongly closed or left open, they can take it to Meta. Of course only a portion of users care enough about their issues to actually do this, but even regarding this, there are very little complaints, e.g., if you compare to other Stack Exchange sites.

In addition, review decisions are public and 10 k users have access to statistics on closure and similar that allows to find problematic patterns.

Apart from this, the community seems to work insofar that it still thrives and we are not drowning in complaints. You may see this as a self-fulfilling prophecy, but if this community weren’t working, we would become subject to natural selection. Nobody forces you to participate if you do not like this.


¹ who passed a small reputation threshold, which exists to avoid the system being gamed and to ward off spam and similar

  • You wrote a defense of your methods not an answer to the question. The question was how do you know you're meeting your goal. Not what do you think is fair. – cba1067950 Mar 20 '17 at 10:52
  • 3
    @cba1067950 The answer to your question is in the last paragraph. – Massimo Ortolano Mar 20 '17 at 11:01
  • 4
    @cba1067950 The specific question on how to figure out whether we meet some standards seems pretty silly to me to be honest. Please do not try to make the site ISO 9000 compliant or whatever the current hotness is. – Tobias Kildetoft Mar 20 '17 at 15:24
  • The answer in the last paragraph is "It works because I like it." The question was "How do you know it is making others feel comfortable?" or "How do you know this is an improvement over other methods?" Or to put another way I asked is there a meta test to check to see if your methods are working. He listed the methods. He didn't list a method to check the methods. @TobiasKildetoft Judging by the quality control here, I'm not surprised you think it is silly. It doesn't serve you it serves others. So naturally it doesn't interest you. – cba1067950 Mar 20 '17 at 19:46
  • @TobiasKildetoft I am happy that you basically said ensuring standards is silly though because it is a direct contradiction to the claim that there are checks to ensure standards. And +2 to boot. This is a perfect example that two contradicting claims can be praised simply because it defends the culture. This will inevitably be downvoted to oblivion but it is good that you confirmed my suspicion. Thanks. – cba1067950 Mar 20 '17 at 20:37
  • @cba1067950: See my edit. – Wrzlprmft Mar 21 '17 at 7:31
  • That's closer but still doesn't answer the question. The results of the votes doesn't ensure that the votes are properly creating a superior environment. An addition test doesn't measure calculus aptitude because people get A's. Also popularity of a belief doesn't mean it is accurate. And you're assuming that no complaints means no problems. A company with 0 customers has no complaints. I'm offering a criticism of methods that were debunked hundreds of years ago, to improve the site on Meta, it has been down voted and I'm being told to leave, so this isn't effective either. – cba1067950 Mar 23 '17 at 23:10
  • 2
    And you're assuming that no complaints means no problems. A company with 0 customers has no complaints. – No, I am assuming that few complaints in relation to the number of users/askers (which is not zero by the way) is indicative of no egregious problems. — I'm offering a criticism of methods that were debunked hundreds of years ago, to improve the site on Meta – So far, you only asked how we do self control; I answered that. You did not offer any criticism on our way to do self control beyond what is known, but unavoidable (with reasonable effort). You did not offer any improvement. – Wrzlprmft Mar 24 '17 at 8:04
  • 2
    @cba1067950 And, afaik, no one told you to leave. – Massimo Ortolano Mar 25 '17 at 7:38
  • @Wrzlprmft no complaints was an example. The point is you're assuming because you don't have a test to generate knowledge. The question is critical of the method. Reusing the same test for general questions as a meta analysis assumes the test works. That is not a meta analysis. It is a circular argument. "This test works. How do I know it works? Because I used it to check." – cba1067950 Apr 1 '17 at 23:40
  • @MassimoOrtolano "Nobody forces you to participate." People who don't like it here don't stay, thus create no evidence of dissatisfaction. And no evidence is an illogical proof of the methods. He could have said a generic, "Nobody forces people to participate." Maybe it's a colloquialism but it's more likely he is attaching his bias to the point to mask the fallacy of the point. The downvotes are evidence enough that this isn't sinking in though. – cba1067950 Apr 1 '17 at 23:58
  • @cba1067950: The you in my last sentence was generic. — People who don't like it here don't stay, thus create no evidence of dissatisfaction. – The Internet doesn’t work like this. Sure, for most people it’s true what you say, but there always is a certain percentage of people who invest the time to complain, e.g. you. I tried to address your specific critique (on our closing behaviour) but received no feedback from you. Apart from that, you only complain on a double meta level about things that we cannot reasonably change – or at least don’t know how, and you are not telling us either. – Wrzlprmft Apr 2 '17 at 6:30
  • @Wrzlprmft This isn't metaphysics. The internet is the most easily quantifiable technology ever created. It creates more data than anything ever has. Every day other websites collect feedback and form conclusions about it. I'm just asking an honest question not demanding anyone build a new site. Instead I got a rant about how good the site is. "The methods are x,y,z. No meta analysis above what you see." That's all it needed. – cba1067950 Apr 2 '17 at 8:39
  • @cba1067950: There is a lot of data and statistics available. It’s mentioned in both answers you received. Most of it is accessible to everybody. — Instead I got a rant about how good the site is. – Where? (Please quote at least one sentence to support your answer.) – Wrzlprmft Apr 2 '17 at 9:01
  • @Wrzlprmft Again that data is irrelevant. "Your entire response up to the last section." – cba1067950 Apr 2 '17 at 9:27
1

How to check internal measurements of standards.

During our beta period there were a few site-evaluations when users were asked to rate how we were doing. There is also the data explorer that lets users dig into the AC.SE database to see how things are doing. Finally, diamond moderators and SE staff have access to site analytics that go way beyond votes and page visits. For example, one metric allows us to see where anonymous users are coming from and how they are voting (these votes do not show up in the vote total).

In other words the health of the site is measured on a bunch of metrics at all levels of the game (users, moderators, and staff).

  • Analytics on page visits and votes aren't meta analysis on whether the system works. Your site evaluations appear to be more self grading, which is essentially the same thing as the voting system. And with 20 or so responses it's probably safe to assume it's a self assessment by regulars. So again, not really a meta analysis. But fuck it. You're all pretty hell bent on offering a bad product so I'll just close this out. – cba1067950 Apr 2 '17 at 0:10
  • If you ask me, the site-evaluations were pretty pointless and got abolished for a good reason. – Wrzlprmft Apr 2 '17 at 6:32
  • @cba1067950: So again, not really a meta analysis. – So, what do you propose? How can we reasonably do this? — offering a bad product – we are not offering a product (in the commercial sense) at all. Also, why don’t you start enlightening us about why it is so bad? – Wrzlprmft Apr 2 '17 at 6:37
  • @Wrzlprmft I did ask you and everyone else. And everyone said they existed. And now you're saying they don't and that they were useless. Personally I think everything from the search results, organization, incomplete solutions, vague/poorly enforced/annoying rules, and the assumption that duplicates are redundant needs reworking but without metrics who knows what others think. The system blocks people from contributing feedback because of your spam protocol and defensiveness when you could just give a separate metrics. This site makes money. It is a product 100% in the commercial sense. – cba1067950 Apr 2 '17 at 8:24
  • @cba1067950: And now you're saying they don't and that they were useless. – No, I am saying that the site self-evaluations were useless. I do not say that other mechanisms of self-evaluation that were not explicitly named thusly are useless. – Wrzlprmft Apr 2 '17 at 8:27
  • Personally I think everything […] needs reworking but without metrics who knows what others think. – What does the first part of this sentence have to do with the second one? — The system blocks people from contributing feedback because of your spam protocol and defensiveness when you could just give a separate metrics. – Can you elaborate on this? – Wrzlprmft Apr 2 '17 at 8:35
  • This site makes money. It is a product 100% in the commercial sense. – Academia SE does not make any money on its own – what would be your source of revenue? Stack Exchange Q&A in general makes some money through ads on some site. AFAIK, Stack Overflow Careers is what is the main source of revenue. Either way, we (that is the Academia SE community) do not sell any product. Rather, we are the customers or the product (depending on your point of view). – Wrzlprmft Apr 2 '17 at 8:42
  • @Wrzlprmft The first part is what I would like done to improve the site. The second part is about what others may want. If this was automated you would have one data point in a survey but you would need more. If it is free you are the product. Academia is pulling in traffic and keeps people using the product longer for other things. Depending on how they sell their data to advertisers (hours spent on all SE) this could be generating income. This is a cheap experiment. It may eventually have ads but it increases the overall value. Any work that is being done can be sold. – cba1067950 Apr 2 '17 at 8:50
  • The second part is about what others may want. – “no metrics who knows what others think”? What are you referring to? — If this was automated you would have one data point in a survey but you would need more. – So, what you want/suggest/miss is an automatism to ask users how they liked the site? – Wrzlprmft Apr 2 '17 at 9:01
-9

There appears to be no meta data on the quality control checks.

And all responses appear to be reinforcing the idea that there is nothing checking against self centered expressions of ego in an isolated community.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .