It’s in a peer-reviewed paper, so it must be true. Right? Alas, you can only really hold this belief if you don’t know about the peer-review system, and scientific publishing more generally.

That’s why, in this episode of The Studies Show, Tom and Stuart break down the traditional scientific publishing process, discuss how it leads science astray, and talk about the ways in which, if we really cared, we could make it better.

The Studies Show is brought to you by Works in Progress magazine. Their new September 2024 issue is out now, and is brimming with fascinating articles including one on lab-grown diamonds, one on genetically-engineered mosquitoes, and one on the evolution of drip coffee. Check it out at worksinprogress.co.

Show Notes

* A history of Philosophical Transactions, the oldest scientific journal

* Hooke (1665) on “A Spot in One of the Belts of Jupiter”

* The original paper proposing the h-index

* Useful 2017 paper on perverse incentives and hypercompetition in science

* Goodhart’s Law

* Bad behaviour by scientists:

* What is a “predatory journal”?

* Science investigates paper mills and their bribery tactics

* The best example yet seen of salami slicing

* Brief discussion of citation manipulation

* Elisabeth Bik on citation rings

* The recent discovery of sneaked citations, hidden in the metadata of a paper

* The Spanish scientist who claims to publish a scientific paper every two days

* Science report on the fake anemone paper that the journal didn’t want to retract

* Transcript of Ronald Fisher’s 1938 lecture in which he said his famous line about statisticians only being able to offer a post-mortem

* 2017 Guardian article about the strange and highly profitable world of scientific publishing

* Brian Nosek’s 2012 “scientific utopia” paper

* Stuart’s 2022 Guardian article on how we could do away with scientific papers altogether

* The new Octopus platform for publishing scientific resaerch

* Roger Giner-Sorolla’s article on “aesthetic standards” in scientific publishing and how they damage science

* The Transparency and Openness Practices guidelines that journals can be rated on

* Registered Reports - a description, and a further discussion from Chris Chambers

* 2021 paper showing fewer positive results in Registered Reports compared with standard scientific publication

Credits

The Studies Show is produced by Julian Mayers at Yada Yada Productions.