Are questions on the theory of academic research on-topic?

Question

Working in academia is not just navigating university politics. It is also about conducting research and teaching one's field to students.

Academia already has an explicit policy of accepting questions on "university pedagogy". This covers the teaching part of a university employee's life.

But research is as least as important as teaching. (In fact I know many academics who are willing to do research without teaching, and none who would like to do teaching without research, so maybe it is even considered more important by academics?). So people in academia need to know how research is done.

And I don't mean the nitty gritty stuff of choosing a varimax or quartimax rotation for the factor analysis in a given experiment. This can be asked on stats.stackexchange.

But generations of researchers have created a body of theory of science and philosophy of science. Starting with epistemological vs. ontological definitions of knowledge, and going into different theoretical perspectives and methodologies. This is a topic which is field-independent; every scientist's work is touched by it, no matter whether she is in psychology, molecular biology or material science.

Doing research without this kind of knowledge is like hacking circuits without knowing Ohm's law. I know it, because I've tried doing research that way for the last four years and wondered why I am in a downward spiral of doing something, getting feedback from my supervisor or from reviwers that it is not the right thing, then trying to correct it (less motivated this time), and still not getting it right. Graduate students need to know it, else they do the wrong things. Professors need to know it, so they can explain to their grad students why the wrong thing they are doing is wrong, and how to find the thing which is right.

There is need for this knowledge within academia. The everyday worklife of people within academia is affected by it. Academics are also currently the only people who have that knowledge. The theorists of science are scientists themselves. The professors who guide us, who define daily what good science is by deciding which article to publish and which to reject, by applying for grants for projects which are compliant with scientific principles, by evaluating their grad students' work, they all have an understanding of what proper research looks like, whether on an explicit or on an intuitive level. Experienced academics are the experts on this type of knowledge, and unexperienced ones are very much in need of it, in order to become better academics.

I think that this situation perfectly reflects the spirit of StackExchange as a place where the experts in one topic answer the questions of the people starting out in their area, in order to make this type of knowledge available to everyone who struggles with becoming better in what he or she is doing.

Therefore, I think this should be part of the on-topic areas for Academia.Stackexchange. It is the right place for it, and the community can only gain from allowing it.

---

This post was prompted by the fact that I asked a question on this topic and got several reactions from community members who found it off-topic. I don't know why they think so. Butif it is just because "this is not the kind of question we are accustomed to seeing around", then this shouldn't be a barrier. Just because nobody before has thought of asking this kind of question doesn't mean it isn't interesting for the people who ask it and the ones who answer. (And several people expressed interest too).

I would also like to note that they couldn't think of a site which is better suited, and I think I made it clear in my argument above why this is the community to which it is suited, and not any other.

It is up to the community to decide whether it wants to accept or reject this type of question. I would find it very sad if it decides to reject it, because this site would be its natural home, just like in real life, the university is the home of scientific theory.

Answer 1

I believe the linked question (and other similar questions) is on-topic and if it is not, then it should be. I do not see why we would consider "How to improve oneself as a teacher" or pedagogical / classroom management issues and we would not consider "how to research" or research management issues.

My vote is to keep the linked question and support these kinds of questions in the future.

Answer 2

It's an interesting question and I have no problem with where it should be posted. My approach is always if there is a question, then let's try to answer. (And I walk the walk; I just told a user how to interact with a visa officer). However, I have a hard time with the OP insisting that "this question is for Academia" = "this question cannot be posted on anywhere else." They are fundamentally two things.

I expressed in the comment that the question is suitable for CV, but got immediately shot down by a comment that "CV is absolutely not the right place." I will be very surprised that a site called Cross Validated will reject a technical question about the definitions of validity and reliability.

And as I have said, I am only expressing my comment on the "This question cannot be posted on anywhere else" bit. I believe Academia would have enough users who are familiar with this idea of validity and reliability to answer the question. And hence this is one of the right homes. I was just very surprised by the declaration "This is not all about statistics!! So, it cannot be posted on a statistics board!!" While the question itself is mostly related to definitions, and the scenarios are also applied.

Anyway, I might have misunderstood your motive. Perhaps you didn't want just an answer, perhaps you thought of getting an in-depth discussion on the deeper meaning of this two concepts. My motive is to provide a suggested place for you to get the answer... and the suggested board needs not to be the origin of that knowledge. Just like if I want to know which of the two floor cleaners I should buy, I will have no problem asking a janitor working in our office, and I bet he/she does not have a PhD in chemical engineering (but more wonderful if he/she does). For this reason, Academia is fine, CV is fine.

Answer 3

A clarification on the comment I made under the original question about validity,

CV is absolutely not the right place.

It has created some misunderstanding, because I formulated it badly. It was a hastily written comment, in a moment when I was anticipating my question getting closed before it gets an answer - if you've been there, you know how it feels. Now I wish I had thought more before writing.

Sometimes a question will have a topic which is interesting to more than one type of expert. Then it is up to the asker to decide where to post it. But an experienced asker knows that it will get a different answer, depending on where it is asked, so he chooses the place according to this.

As an example, we had a question on Cooking recently, "determining the sugar content of liqueurs". Now, this is a question a chemist or a physicist can answer too. And indeed, after some discussion in the comments, I posted a question on Chemistry.SE, because I wanted to know which existing measuring tool is better, density or refraction based.

But I think that it was a very good decision on the part of the OP to ask on Cooking first. Because we were able to tell him that, while there are tools for measuring the sugar content, they won't let him solve his original problem of deciding how much sugar to add to alcohol-containing truffles, because sweetness perception depends on much more than just sucrose content. I think that a cook's perspective was more helpful in this case than a chemist's perspective.

Similarly, when I asked my question on Academia, I already knew that I am interested in a theory-of-science perspective. And that while there surely are statisticians who are well-versed in theory of science, I figured that I have a much better chance of getting this type of answer on Academia. So my comment should have been worded something like "please don't migrate it to Crossvalidated, because for this specific question, I know they will not give the kind of answer I am interested in." I didn't mean to suggest that questions about validity are generally not a good topic for statistics, although now that I re-read the comment, I understand why many people thought that I am saying that.

I agree with Penguin_Knight that in general, Crossvalidated is one of the right homes for questions on validity and reliability. I created the current meta question because I think that Academia is another one, (and because there are other questions on scientific theory not concerned with validity), not because I think that it is the single right place for questions of this type.

Answer 4

I mostly agree with @Penguin_knight, but want to add a slightly different direction:

• I'm active both on academia and on cross validated.
  I'd have given the same answer to the same question regardless where it ended up.

• As the question is worded, I'd have agreed that it should be moved to cross validated. Not because validity and reliability are not relevant in academia, but because cross validated is the more specialized forum where it is appropriate. That is, it is more on-topic there.

• There is one very important point in the meta question here that is relevant for the decision where the question is most on topic: you explain here that you are interested less in an explanation of what exactly constitutes validity and reliability than on the general implications for research. I think if that had been spelled out in the question, the whole move-to-crossvalidated-or-not discussion would have been avoided.

• I'm still slightly in favor of cross validated as the most appropriate place. My reasoning why academia is not automatically the right place even if the intended point of view will be more than just statistics is that validation is far from being an exclusively academic question. It is of huge importance in industry.
  Maybe not the most compelling reason, but a slight indication: the answers you got so far could perfectly stand on cross validated as well.

Answer 5

I don't agree with the premise of this question. The information about theory and philosophy of science, including the linked question, seems to me to be utterly irrelevant to day-to-day life of many academics. A more apt comparison than that provided in the question would be to hacking circuits without knowing Maxwell's equations, which is perfectly reasonable.

Of course, questions about how grad students can better understand why they are doing things wrong, or about how professors can better explain to grad students why they are doing things wrong, or so on, should be on topic. I have no objection to that. But I can't see how the kinds of theoretical/philosophical topics that this meta question is about fit that description.