On answers to reference requests that do not provide references

Question

I have noticed several instances (and there have been a few flags) where the question says quite clearly something like:

I am looking for references about [some topic]. I'm not really interested in anecdotes or opinions; I am looking for data or studies.

And yet the top answer (often quite upvoted) is:

Well, I don't have any references, but I've observed this too. One time, there was an [anecdote].... It seems to me that [opinion and speculation]....

To me, this is particularly annoying when the asker states that they are an "expert" (e.g., the reference request is based on an observation from many years of teaching) and the reply is "common sense" (e.g., a student's opinion or reasoning).

Relying on downvotes to handle these answers does not seem to be working (particularly on popular questions where far more people have the upvote privilege than the downvote privilege). And to be fair, often there are no studies that anyone is aware of, and so if we delete all answers that don't contain solid references, the question will go unanswered.

So: I am just trying to get a sense for how people feel about this. Is this a problem/annoyance/sub-optimal behavior that we should move towards discouraging or disallowing? Or is it fine and there is nothing to see here? I'll add some voting options to make this slightly more concrete, but feel free to add your thoughts in an answer.

Answer 1

Downvote and add a comment to make it clear what the problem with that answer is.

Answer 2

These answers should not be allowed. We already have flags for "not an answer", and a response to a reference request that doesn't provide a reference is certainly "not an answer" to the actual question. Thus, under existing policy, such answers should be deleted. There is no need for "zero-tolerance" -- flaggers and mods can handle cases individually -- but in most cases, such answers should be deleted.

Answer 3

These answers should not be allowed, but we should not delete them retroactively and for new questions we should first notify the post author with a post notice.

In case we decide to no longer allow these kind of answers, I think it would be unfair to the answerers of the already existing questions to delete their answers after the policy has become effective.

For newer questions, I think that before deleting an answer, moderators should invite the author to add references by adding the following post notice and wait a few days before deleting:

Answer 4

The appropriate response depends on the "hardness" of the reference request, which depends on how the tag is interpreted.

• "Hard" reference requests give clear reasons why an experience-based answer is not acceptable, as in this well-received example. In the case of a "hard" reference request, an answer without a reference is not an answer and should be downvoted and/or deleted.
• "Soft" reference requests bundle "have there been any studies?" in with a larger question that can often be answered without references, such as this example where my answer was accepted with no references. In this case, references are optional and reference-free answers are entirely reasonable to consider.

The distinction between "hard" and "soft", however, is a matter of interpretation and may change as the question is edited.

Answer 5

Questions which request references should be checked to see if they question is about the "content of research." If it is, then the long-standing practice is to close the question.

Answer 6

Note that many questions on the site have partial answers and even answers that don't, precisely, follow the instructions of the OP. Too many questions, IMO, are of the Yes/No variety, so, technically, a one word answer would follow the OPs instructions, and be useless.

But the site isn't intended as a resource pool like a library is. It is a site that offers advice to academics on their questions and often enough the question they ask are subtle enough that explanation is needed, even redirection. Academics have misconceptions like anyone else.

I'll admit that I'm one of the "offenders" here. I try to label my answers as partial or advisory when they aren't technically answers. For such resource requests I might answer when I think that the existence of the resource is very unlikely but that orthogonal thinking might resolve the OPs need for the resource.

I think we are fine without disadvantaging such answers. The OP can ignore them, and others with similar concerns might benefit from them.

Answer 7

These answers should be decided by upvotes/downvotes, as now. Upvotes and downvotes will reward and punish the best answers. Often the answer to a reference request is "there are no references," and so empirical answers are the best we can do. Of course, flaggers and mods can decide on cases individually -- but in most cases, we should not delete such answers.