Discussion about Academia Community Polls

Question

Please discuss any ideas and issues about Academia Community Polls in this thread to keep it tidy.

Answer 1

Two recent poll questions have been added by @user8005.

Do you agree that evolution is the best explanation for the origin of human life on earth? and Do you think there's a purpose of life?.

I am not sure what the purpose of the poll questions are, but these seem so far off topic that I think they should be deleted.

Answer 2

The phrasing in "Are you currently working in Academia (i.e., employed by a university with a primary duty of performing research)?" is overly narrow in excluding anyone whose primary duty isn't research. This amounts to saying many faculty members aren't academics at all, for example those in liberal arts colleges, teaching-focused comprehensive universities, community colleges, etc., as well as many adjuncts or teaching faculty in research universities.

I'm not sure how to fix this, given that we don't know whether the answers so far are based on the narrow interpretation or not, but it's worth keeping in mind in the future to avoid causing offense to academics who aren't primarily researchers.

Answer 3

Regarding the question "Are you currently working in Academia (i.e., employed by a university with a primary duty of performing research)?"

I'm not sure how a student should answer this one. If s/he gets the salary form the university? If s/he is on fellowship?

Answer 4

The polling format does not let people change their answers when their status changes.

The following questions for example have answers that vary in time:

Which of the following "roles" describes you best?

How many publications in peer-reviewed journals have you made in your career?

Are you currently working in Academia

This one, without going into the details of how "happy" is an ill-defined descriptor, probably has an answer that varies frequently:

Are/were you happy in your PhD program?

People can up-vote another answer, duplicating entry, but not cancel their previous votes.

Answer 5

Well, since we are a site of academians here is my curmudgeonly take on polls like these. Feel free to upvote/downvote or discuss.

Problems with interpreting polls like these

• It isn't a random sample of the population of academics, so inferences about the distribution of responses for Academics in general can not be made, under any reasonable circumstances.
• Even questions that focus specifically on the population that participate here on the Academia site, it isn't a representative sample, so inferences to even the characteristics of individuals who use the site are very questionable.

It is fairly simple to illustrate the problematic aspects. Those more active on the sight are more likely to participate in meta. Thus non-response in polls is not a random sample of the individuals on the site. Some might argue this is a good thing (if you are more active you should be given more weight), but to suggest this results in any reasonable inference is tantamount to saying two wrongs make a right.

This doesn't even cover other problematic aspects of voting on the comments that may prejudice the results, such as comments that are placed first have higher exposure, and after accumulation of so many comments some are collapsed from the view. Both these circumstances will be likely to bias polls to already up-voted answers, making long term collection of the polls potentially even more sytematically biased.

I don't deny such poll questions can be fun and intriging, but lets be serious about what useful information that can be gleaned from such things. Potentially more useful ways to carry out some of the analysis that motivated the poll question are to utilize the Stack Exchange data explorer (when we get out of beta), or to just scrape info. from user pages. I realize some of the questions are not possible from here, but that isn't a reason to pretend like comment polls are a reasonable solution.

For instance, in regards to whether we are CS centric, it wouldn't be trivial to scrape the data, but if you just go to user pages you could look at the reputation on other sites. You could then look at the distribution of users here to determine if people from either of the computer science sites post a disproportionate number of the questions/answers here. In terms of location data that would be pretty simple when we are out of beta (see a similar question and answers on the stats site).

(For fairness these are both problematic as well, for people don't need to connect there accounts and people don't need to supply location information, but again, problems with these approaches don't justify innapropriate polling!)

---

To place the distinction between these polls and generally interpreting upvotes/downvotes anywhere on the StackExchange sites, the (typical) goal is to ask a question and receive an answer. Votes (and checking as answered) are generally taken as tokens of the correctness/usefulness of the answer to the original question to outsiders.

While upvotes/downvotes on any question are similarly suspect to critiques of making inferences, any answer can be evaluated in absentia other context by a reader and determine its usefulness. That is, the poll of up and down voting is not the goal of asking a question, and if the site blinded them tomorrow the questions and answers currently on the site would not suffer from it.

The question that these polls attempt to measure is merely what proportions of the community conform to certain categories. Usefulness, should (I hope) be measured in how accurate an estimate they provide. Here, the upvotes and downvotes are the answer, and I've already provided reasoning as to why they aren't likely to be a very good answer to the question.

Answer 6

The time zone poll got a bit confused, because it appears that some people answered based on daylight saving time (summer time) for their time zone, whereas others answered based on standard time.

Answer 7

Doing little clean up of the poll page. There was a question regarding this comment/answer

Academia Community Polls

Which adds a non-binary gender type. I think it is covered at http://gender.wikia.com/wiki/Non-binary