This is a follow up to the idea collection, please see that post as to what this is about.
First of all, thanks to everybody who participated in the idea collection. I tried to distill the results into a coherent set of guidelines, filling some apparent gaps here and there. The result can be found below.
In general, we will continue to give wide latitude to askers, answerers, and commenters; this is not an attempt to introduce many new, confusing rules. However, introducing these guidelines will be helpful in order to mediate disputes or other cases where moderation action becomes necessary.
Please review the guidelines and do the following:
- Feel free to edit minor issues directly. This is a community wiki.
- Post an answer if you disagree with a point, want to propose an addendum, or similar. However, please refrain from simply re-iterating suggestions that were already rejected by votes in the idea collection.
How to deal with a question that contains a false premise?
Somebody posted a question that makes an assumption I disagree with or think might be wrong. How do I best inform the asker of this? Do I post a comment or an answer, or should I vote to close the question or do nothing at all?
What you want do is called a frame challenge: You claim or suggest that the question is based on a misconception, wrong assessment of a situation, or similar. The frame set by the asker is challenged by the answer or comment you want to post.
The following is mostly a set of guidelines that should give you an idea of best practices and avoid unnecessary confrontation. There often is a lot of leeway, but drastic deviations will be moderated.
We primarily distinguish frame challenges by what they are about, with some general rules of thumb at the end:
Misconceptions about academic procedures, norms, or similar
These misconceptions concern the very topic of this site. For example:
Assertion: You must have a PhD to submit a paper to a journal.
These are mostly free game for frame challenges. However, before challenging such an assumption, please consider that you may be wrong because academic customs vary a lot between fields, countries, etc..
Misconceptions about off-topic aspects
Typically this is about the content of academic research or teaching, but it may also be other off-topic things. For example:
Assertion: My newly developed method that makes a very good guess whether a number is prime topples modern cryptography.
Whether such statements are correct is off-topic here. If you want to discuss this off-topic material, do it in chat; discussion in the comments or answers will likely be deleted. If such an off-topic aspect is central to a question, the question should likely be closed or migrated.
Wrong goal (XY problem)
The asker wants to achieve X and thinks doing Y helps them to do this, so they ask how to do Y. However, Y is not a good way to achieve X.
Most often, we can only suspect an XY problem, since the asker doesn’t talk about X, but only about their outlandish goal Y. In this case, we can only tactfully inquire what the asker wants:
Can you please elaborate why you want to do this, so we can provide better answers? I may be wrong, but this feels like an XY problem.
If X is detailed, you can suggest alternative ways to achieve it, as long as the focus is helping the asker.
Wrong assessments of individual experiences
Here the asker potentially wrongly assessed something that happened to them – as opposed to general facts. For example:
Assertion: The student I supervise does not take my criticism seriously.
Assertion: My paper was cited for some claim it did not make.
In these cases, we almost always lack all the relevant information (or it would be off-topic) and cannot make a judgement. The asker should know better than we do, and they are responsible to ensure that such an assertion is correct. Therefore frame challenges about such situations are usually not appropriate.
However, there are some exceptions, where a short and tactful caveat is appropriate:
The misconception is common and applies to many people in a similar situation, e.g.:
Assertion: The referee did not thoroughly review my paper, as they misunderstood the key concept.
Caveat: Before proceeding please consider that you are very familiar with your work and thus may not have noticed shortcomings in your explanations.
The asker describes in detail how they arrived at an assertion and this makes it seem unlikely that they are correct. For example:
Assertion: My professor is not satisfied with my work, because X, Y, and Z.
Caveat: What you describe are normal activities for a supervisor. Just by your report I would not assume that your professor is dissatisfied.
If the asker’s judgement should be incorrect, it may have severe consequences:
Assertion: My professor asked me to fudge some data by applying X.
Caveat: Please be aware that this is a serious accusation. I am not saying you are wrong, but before escalating this, please consider consulting with an expert whether applying X is really inappropriate in this situation.
In all such cases, such a caveat should not be much longer than the asker’s description of the assertion and respect the asker’s assessment instead of directly denying it.
Mind that this does not apply to questions asking us to evaluate a situation, e.g.:
My professor does X, Y, and Z. Is this normal? Does this mean that she is not satisfied with my work?
Sexism, racism, discrimination, and other traumatic events
A relevant subcategory is when the asker experienced sexism, racism, discrimination, or similar behaviour, usually towards themselves. Such events are often traumatic and denying what happened may easily add to the trauma. Moreover, we almost certainly don’t know all the details (context, tone, gestures) and thus cannot judge the situation.
In this case, the above exceptions do not apply: We can assume that the asker has already considered alternative interpretations of events and is aware of the severity of the respective accusations. At best, you may very tactfully ask for further details or assess the details if relevant for the question, e.g.:
I am sorry for your experience. To better answer your question, can you please  your question to tell us whether you have any evidence of this? I understand if you do not want to go into the details; it suffices to know how much evidence you roughly have.
General rules of thumb
Before you write a frame challenge, see whether you can answer all of the following with yes:
Is the misconception central to the question? If the question can be asked as well without the misconception, it’s better to edit it out or only address it briefly. If on-topic, you can ask or suggest a separate question about it.
Would you write a frame challenge if the question provided fewer details?
Are you confident that the asker did not already consider your frame challenge?
Does your frame challenge actually help the asker?
Does your frame challenge respect the asker, in particular their expertise, privacy, and problems?