2

Please discuss any ideas and issues about Academia Community Polls in this thread to keep it tidy.

1
  • I have been told in direct response to my poll that only demographic questions are appropriate in this community poll. Is this the intention of the Academia Community Poll ? Can someone please clarify this in the rules section. Apr 14, 2015 at 19:20

7 Answers 7

7

Two recent poll questions have been added by @user8005.

Do you agree that evolution is the best explanation for the origin of human life on earth? and Do you think there's a purpose of life?.

I am not sure what the purpose of the poll questions are, but these seem so far off topic that I think they should be deleted.

2
  • The two questions have been deleted.
    – user102
    May 5, 2013 at 12:19
  • @DanielEShub I don't check meta very often. I would not answer user8005's question on the main site and chat room if I had seen this comment earlier. Obviously, he is not someone I would like to be talking to. Thanks for alerting it.
    – Nobody
    May 6, 2013 at 9:13
6

The phrasing in "Are you currently working in Academia (i.e., employed by a university with a primary duty of performing research)?" is overly narrow in excluding anyone whose primary duty isn't research. This amounts to saying many faculty members aren't academics at all, for example those in liberal arts colleges, teaching-focused comprehensive universities, community colleges, etc., as well as many adjuncts or teaching faculty in research universities.

I'm not sure how to fix this, given that we don't know whether the answers so far are based on the narrow interpretation or not, but it's worth keeping in mind in the future to avoid causing offense to academics who aren't primarily researchers.

3

Regarding the question "Are you currently working in Academia (i.e., employed by a university with a primary duty of performing research)?"

I'm not sure how a student should answer this one. If s/he gets the salary form the university? If s/he is on fellowship?

3
  • 2
    I think all students would answer yes, independent of their funding.
    – StrongBad
    Apr 5, 2013 at 7:14
  • Didn't think about this. I agree with @Daniel, I would definitely say "yes"; your primary duty is research. This also implies that non-research masters students (i.e., professional masters) would answer "no".
    – eykanal
    Apr 5, 2013 at 12:31
  • 1
    I would expect all students to answer "no" because they are students foremost. That is, I read the question in a way that "working" did not include students. Perhaps this is because I did not think of myself as "working" even when I had a graduate assistantship ... Dec 6, 2015 at 23:21
3

The polling format does not let people change their answers when their status changes.

The following questions for example have answers that vary in time:

Which of the following "roles" describes you best?

How many publications in peer-reviewed journals have you made in your career?

Are you currently working in Academia

This one, without going into the details of how "happy" is an ill-defined descriptor, probably has an answer that varies frequently:

Are/were you happy in your PhD program?

People can up-vote another answer, duplicating entry, but not cancel their previous votes.

1
  • 1
    Given the limitations of the SO platform for this kind of pollings, should we be really interested in their results, we can think of resetting the results once a year. Dec 9, 2015 at 19:45
2

Well, since we are a site of academians here is my curmudgeonly take on polls like these. Feel free to upvote/downvote or discuss.

Problems with interpreting polls like these

  • It isn't a random sample of the population of academics, so inferences about the distribution of responses for Academics in general can not be made, under any reasonable circumstances.
  • Even questions that focus specifically on the population that participate here on the Academia site, it isn't a representative sample, so inferences to even the characteristics of individuals who use the site are very questionable.

It is fairly simple to illustrate the problematic aspects. Those more active on the sight are more likely to participate in meta. Thus non-response in polls is not a random sample of the individuals on the site. Some might argue this is a good thing (if you are more active you should be given more weight), but to suggest this results in any reasonable inference is tantamount to saying two wrongs make a right.

This doesn't even cover other problematic aspects of voting on the comments that may prejudice the results, such as comments that are placed first have higher exposure, and after accumulation of so many comments some are collapsed from the view. Both these circumstances will be likely to bias polls to already up-voted answers, making long term collection of the polls potentially even more sytematically biased.

I don't deny such poll questions can be fun and intriging, but lets be serious about what useful information that can be gleaned from such things. Potentially more useful ways to carry out some of the analysis that motivated the poll question are to utilize the Stack Exchange data explorer (when we get out of beta), or to just scrape info. from user pages. I realize some of the questions are not possible from here, but that isn't a reason to pretend like comment polls are a reasonable solution.

For instance, in regards to whether we are CS centric, it wouldn't be trivial to scrape the data, but if you just go to user pages you could look at the reputation on other sites. You could then look at the distribution of users here to determine if people from either of the computer science sites post a disproportionate number of the questions/answers here. In terms of location data that would be pretty simple when we are out of beta (see a similar question and answers on the stats site).

(For fairness these are both problematic as well, for people don't need to connect there accounts and people don't need to supply location information, but again, problems with these approaches don't justify innapropriate polling!)


To place the distinction between these polls and generally interpreting upvotes/downvotes anywhere on the StackExchange sites, the (typical) goal is to ask a question and receive an answer. Votes (and checking as answered) are generally taken as tokens of the correctness/usefulness of the answer to the original question to outsiders.

While upvotes/downvotes on any question are similarly suspect to critiques of making inferences, any answer can be evaluated in absentia other context by a reader and determine its usefulness. That is, the poll of up and down voting is not the goal of asking a question, and if the site blinded them tomorrow the questions and answers currently on the site would not suffer from it.

The question that these polls attempt to measure is merely what proportions of the community conform to certain categories. Usefulness, should (I hope) be measured in how accurate an estimate they provide. Here, the upvotes and downvotes are the answer, and I've already provided reasoning as to why they aren't likely to be a very good answer to the question.

6
  • In regards to population. The comments I was concerned about where "are the people who vote to close questions US/CS centric". So I think a meta poll probably samples the correct population. I don't know enough about the comment sort order to know how it affects bias, but I am sure it does.
    – StrongBad
    Apr 5, 2013 at 16:55
  • 1
    There sure are many biases, but I think: 1. it's better than nothing (data and user page scraping has many (more?) issues); 2. it gives a good feeling of the equilibrium of the heavily involved part of the community… if we keep that in mind, it is a useful tool.
    – F'x
    Apr 5, 2013 at 17:28
  • @F'x I don't see how data and user pages has more issues. I can understand the sentiment that it is better than nothing, but lets interpret for what it is. Polling en masse will not be a reasonable tool to make inferences about the population, and can potentially be misleading. Scraping data from user pages has a direct advantage in that you can tie it to actual responses. For example, Jeffe's vote can only count once in a user poll - but he answers so many questions that any question asking about user contribution to the site would be interested in such info.
    – Andy W
    Apr 5, 2013 at 17:40
  • @DanielE.Shub - I don't even see how your meta question addresses that question - you ask for everyones field and academic systems you are familiar with. People voting on these aren't necessarily the people voting to close questions! All of these polls are merely an guess that they capture some reasonable information from the population.
    – Andy W
    Apr 5, 2013 at 17:47
  • 2
    I don't understand. Are you arguing that the poll is biased because it's not a uniform sample (so my votes count too much), or that it isn't weighted by response frequency (so my votes count too little)? Are you perhaps mistaking a poll on a web site for a scientific instrument?
    – JeffE
    Apr 6, 2013 at 14:28
  • @JeffE, it is a bit of cognitive dissonance to admit the poll isn't a reasonable instrument to measure user characteristics of the site and then forge ahead anyway pretending it has some other utility. I don't view answering questions on here any differently than I do giving colleagues or clients advice. I sure wouldn't condone such a poll in any other circumstance, so why should I pretend like it is ok here?
    – Andy W
    Apr 7, 2013 at 3:19
1

The time zone poll got a bit confused, because it appears that some people answered based on daylight saving time (summer time) for their time zone, whereas others answered based on standard time.

0

Doing little clean up of the poll page. There was a question regarding this comment/answer

Academia Community Polls

Which adds a non-binary gender type. I think it is covered at http://gender.wikia.com/wiki/Non-binary

1
  • I think the gender question should just be deleted altogether instead of trying very hard to make it politically correct. It's completely irrelevant anyways.
    – Cape Code
    Dec 7, 2015 at 8:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .