Is Most Published Research Wrong?

What Hacker News said

Strong rec r_singh · 2018-02 · on “Neural nets more accurate than humans at detecting sexual orientation in images”

If the test data does not have a 50/50 split (or something around that ratio), the headline is straight up lies. Skewed test data is the most common problem with research, reminds me of this great video by Veritasium (Is most published research wrong?): [this video]

Strong rec sillysaurusx · 2020-10 · on “Actively speaking two languages protects against cognitive impairment”

Cue “p = 0.05 is a random choice” and a big thread about p-hacking. Any takers? It feels like everything to say has already been said about the bias of small sample sizes and the dangers of p-hacking. But it also seems like the discussion is inevitable here. I wonder if there’s a new insight to glean, somehow... For a background on why p-values are misleading, see [this video] which is excellent. He points out that 0.05 was selected by Ronald Fischer in 1925, and people have gone with it since.

Strong rec scotty79 · 2021-08 · on “The Coronavirus Is Here Forever”

Please watch this video. It neatly shows why should you treat small and less rigorous studies with suspicion, even if there are many giving some results: [this video]

Strong rec KK7NIL · 2025-12 · on “I don't think Lindley's paradox supports p-circling”

Wikipedia is infamously bad at teaching math. This Veritasium video does a great job at explaining how such skewed priors can easily appear in our current academic system and the paradox in general: [this video]

Recommends burntrelish1273 · 2017-04 · on “Receding glacier causes Canadian river to vanish in four days”

Definitely. Another good idea is to show the difference between reality and data can be overwhelmingly significant. Videos demonstrating this with examples: - "The Bayesian Trap" [LINK] - "Is Most Research Wrong?" [this video]

Recommends ReligiousFlames · 2017-05 · on “Science to Retract Study on Fish and Microplastics”

There's an interesting video by Veratasium "Is Most Published Research Wrong?" (Clickbaity title to be sure.) It shows, on the overall statistics of valid/invalid vs. published/unpublished papers, the arbitrary p-value threshold is wholy inadequate to prevent a sizable fraction of published papers from being false positives (invalid but published). [this video]

Recommends wolfram74 · 2017-07 · on “The US government is removing scientific data from the Internet”

It's not 'these days' it's the intrinsic nature of false hypotheses outnumbering true hypotheses by an enormous ratio. Even if everything were working perfectly you'd expect false results to outnumber true results. It's like the common example of disease screening, where even if you only get a false positive 1% of the time, but there are only 1% of true positives in your sample population, you're going get many more inaccurate results than accurate results. This video walks through an example calculation and what kinds of steps are being taken to improve on things. [this video]

Recommends hannasanarion · 2018-11 · on “Dead sperm whale found in Indonesia had ingested '6kg of plastic'”

My dude. Please go get yourself some education on how science works. I feel embarrassed for you. The reproducibility crisis is about the prevalence of false positive results in the scientific literature. This video is a good primer: [this video] A false positive is when a paper says "this works" when it actually doesn't, because of a statistical fluke in their sample. Even if citing the reproducibility crisis as an excuse to believe something that science has repeatedly found to be false isn't a really bad reason for belief (which it definitely is)... it doesn't even apply because there can't be false positive results regarding neurolinguistic programming because there aren't any positive results