12

A few weeks ago I have asked a question that I believe is about a major challenge that academia faces today and will increasingly face in the future (I deleted it since then because of the reactions it got):

How can we fight junk science? [closed]

The question was closed as unclear but I think that question is pretty clear and there are some answer-worthy comments with very practical points.

Is there a political correctness component or something else that I'm missing?

  • 2
    I didn't understand the reason for closure either, nor the downvotes. The only possible objection, I think, is that it might be too broad, but definitely not unclear. – Massimo Ortolano May 3 '16 at 11:51
  • 1
    Just my perspective, but your question reads like 3/4 rant and 1/4 question. For example, the statement, "Legitimate scientific literature is being drowned in an ocean of junk journals and articles" is strong, backed with no examples, and seems to insist that any answer first agree with the claim. In other words, you're begging the question. Maybe, in the spirit of science, include your colleagues by first asking if others have noticed this, if they agree, and so on. – gwg May 16 '16 at 16:56
  • 1
    @gwg the examples are everywhere. That in particular journalists often pull out bad substandard aka junk science papers that contain spectacular claims not backed up properly by scientific arguments to make loud stories in popular media, is a well known fact these days. And depending on the field, the bad papers overselled in popular media are only the tip of the iceberg. The question is a clear statement of this problem and asks what can be done about it. It is not a "rant". – Dilaton May 17 '16 at 15:41
28

I think there might well be a good question in there, but right now, it's way too broad and poorly defined. Some issues that I see are:

  • It's not clear whether the question can be answered per se, or whether in the current state of the world it's more a matter of discussion, debate, and experiment. This makes it like an "I would like to have a discussion about..." question, which of course are off topic.
  • The definition of the focus is very broad:

    "junk science I mean pseudo-science (i.e. work done with flawed or frivolous methodology), deliberately faked results, and generally very low quality research"

    The issue that I have here is that pseudo-science, fraud, and general boring crap appear to arise in different ways, and attempting to address them all together is extremely complex.

  • The motivation of the question is unclear. Are you more concerned about punishing "bad scientists" or being able to find what you're looking for or about some sort of general societal collapse?

I would thus suggest this might be best addressed not as a single question, but as a collection of more focused questions that tease out particular aspects of the more general topic that are narrow enough to possibly be answerable.

  • 1
    These are interesting suggestions. In fact it's true that the issue is, sadly, too broad for a single question. I'm not interested in "punishing" anyone, unless you take rightfully excluding people from scarcely funded grants and positions a form of punishment. – Cape Code May 3 '16 at 13:40
  • 15
    The first item was the biggie for me - this may or may not be an important question in science today, but it is most certainly not a question that any of our member can be expected to have a clear and "correct" answer for. It's the prototype of a "discussion" question, for which SE is just not the right place. As such, I am also not sure that the question is salvageable while keeping it's spirit intact. – xLeitix May 4 '16 at 10:47
9

Because it's so broad and vague. It could mean one, some or all of sixty things:

  1. how can we individually avoid committing bad science?
  2. how can academics avoid... peers committing bad science?
  3. ... their department committing bad science?
  4. ... their university committing bad science?
  5. ... fellow members of professional societies committing bad science?
  6. how can we advocate for doing some/all of the above?

Your question also mutates. You start by saying a) "junk journals and articles... pompous titles, pay-to-publish [etc.]" but then you say b) "by junk science I mean pseudo-science (i.e. work done with flawed or frivolous methodology), deliberately faked results, and generally very low quality research".

These are each very different things, some of which have no answers, some of which have clear well-known answers. There will always be bad or VLQ research. Peer-review can prevent people publishing e.g. bad physics in recognized physics journals, but it totally can't prevent people splintering off and forming a new field/journal/conference. Grant applications are another thing too, and each grant source has different criteria, levels of rigor, punishment for bad behavior (or lack of). (Peer-review is not perfect either: it's slow, political, fallible and ridden with cliques.)

"Deliberately faked results" constitutes academic fraud, but it's comparatively rare, and I submit to you that many papers simply have no results at all, which is again bad but in a different, passive, way, since it obscures scientific method - unless they're review papers, which they're generally not. There is no shortage of such papers, in most fields.

There are also authors who crank out 10+ papers on essentially the exact same finding - would you call that VLQ? Then again, given reject rates and unpredictable backlogs, who can blame them...?

"Pay-to-publish" is again bad in a different way, not per se, but because it sidesteps peer review, disclosure, replicability, standard use of terminology, which are again all cornerstones of scientific method, so it generally results in a tsunami of crap, and cliques who manufacture plausibility by citing each other.

So, you asked at least sixty different and contradictory questions. The tl;dr is clearly we can't prevent someone setting up the Abkhazia Open Institute of Antigravity and spewing out junk, but we can monitor and publicize any misdeeds, and try to limit them getting access to serious funding. But you knew that already. There is no blunt hammer to prevent the rest of the things you list. Wherever there are economic incentives, however slight, people will respond to them...

  • 1
    A lot of very good points, a bit depressing though. Thanks for taking the time to write that answer. – Cape Code May 9 '16 at 7:11
3

After reading it, I can tell you exactly why.

According to the article "What questions can't I ask here" in the help menu, there's this powerful quote:

To prevent your question from being flagged and possibly removed, avoid asking subjective questions where …

  • .. [four others removed] ..
  • your question is just a rant in disguise: “______ sucks, am I right?”

Simply put, your question is a rant in disguise which is specifically on the do-not-ask list.

That's not to say the thing you're ranting about isn't a problem. But the question that is linked here doesn't appear to be a answerable question (at least, not in this venue.)

  • 1
    Nope-1: Junk science getting too widespread is depending on the field a serious problem these days. Therefore it should be possible to adress the issue here. And indeed, the question is well received by the majority of the mainpage cummunity. If the rules are such that well received questions get closed against the will of the academic community here, these rules should be improved. BTW calling posts you personally dont like or disagree with a rant, is not exactly a good style of discourse ... – Dilaton May 17 '16 at 15:27
  • 3
    I agree junk science is a plague that needs to be abolished. I'm simply saying the question under consideration here isn't really a question. Consider this: how would the true "answer" be decided to that question? It's a very valid rant, and indeed it appears many people on this site agree it's a bad thing. But those votes aren't "This is a great question for our QA format." Instead they're "I stand with you against junk science". I don't want to smother your enthusiasm to campaign against junk science, I'm simply saying the QA format Stack Exchange has isn't designed for this style question. – corsiKa May 17 '16 at 18:15
  • @Dilaton And, for the record, I'm not saying "it was a rant" as an insult. I'm saying that was the conclusion I reached after I read it - purely factual. Simply put, the question is about "the individual and institutional level" but it never talks about the individual or the institution - the only background info in the question is about the overarching problem. Hence why I used the quote "____ sucks, am I right?". I hope this clears up any confusion around my answer here, because I'm not out to insult anyone - I just have an in-depth knowledge of SE's policies and wanted to share insight. – corsiKa May 17 '16 at 18:20
  • there is nothing in the software that in principle prevents the academics here from adressing any topic that is of importance to them. IMHO the only rule that is really needed is that what the community thinks is of interest is on-topic, and what is not is off-topic. That the SE company so strongly interfers with what individual communities are allowed to adress and in particular that on Academia rather the SO model of moderation is adopted than the more academic MO model seems unfortunate to me. – Dilaton May 17 '16 at 20:26
  • 2
    I don't understand your point. The question isn't off topic, it's off format. And there's a good reason why - the 'question' at hand isn't a question as much as it's an invitation to discussion. A question for what an individual can do without describing that individual or the specific situation that individual is facing simply cannot be answered. Those kinds of 'questions' end poorly. That's why they aren't allowed. Consider it collateral damage in exchange for having an orderly system. – corsiKa May 17 '16 at 21:47
  • 1
    So you think not everyone agrees that junk science is a problem? I had the impression that this warranted no discussion. – Cape Code May 18 '16 at 7:07
  • 3
    While I feel the phrase "this warrants no discussion" is dangerous in any case as it leads to non-objective results, no that's not what I was saying above. Rather, I was saying the question invites a discussion about "What can a person do about it?" which is not a good fit for Stack Exchange, particularly when the question at hand contains no information about the person or the junk science they face. The question is basically "awareness raising" which, while noble, is expressly discouraged (as per the "no rants" quote.) – corsiKa May 18 '16 at 14:53
1

I think the answer you wrote somewhere else applies here as well:

The voting rules aren't very strict (as opposed to closing for example) but I don't think the fact that a question is hard to answer is a reason to down-vote. A question should "show research effort be useful and clear" to warrant an up-vote but it's also a matter of personal interests.

I don't think that the issue with the questions you mention is that they are "hard" but rather that they look very much like advertisement for your opinions and pet peeves* that you tried too hard to make look like questions.

My guess is that some users doubt that you genuinely think there are possible answers that would fit this site's format but rather hope for extended discussions in comments supporting your opinion.

Sometimes the click-bait works and you gather many votes, sometimes it's too obnoxious and the opinionated undertone triggers down-votes.


*Ok, so you dislike that some people pay to read articles. We get it.

Ps. Many of your other questions are fine in my opinion.

I think that your question on junk science is much more important than the vast majority of questions on this Stack Exchange website. However, my own experience in posting questions and getting >100 of downvotes is that questions pertaining to improving the research system are often not well received.

I wish questions on the research system were posted on a different Stack Exchange.

  • 1
    Said question was closed but has a positive vote score, so I don't see how that applies. – Cape Code May 13 '16 at 4:41
  • 1
    @CapeCode same in the other thread. And regardless, the same arguments apply. – Franck Dernoncourt May 13 '16 at 5:08
  • 1
    Some questions on the research system might be on-topc here. In particular, when making scientific processes more open, junk science might be an issue that should better be kept under control too ... – Dilaton May 14 '16 at 13:30
  • 2
    I don't see how the arguments could apply. Unless you think there is no consensus that junk science is a bad thing, and thus that would be an opinion of mine. – Cape Code May 18 '16 at 7:05
  • ---------> pet peeve – Franck Dernoncourt May 18 '16 at 20:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .