Nullius in Verba Episode 68: Fraus P-Valoris - I
Oct 10, 2025
This engaging discussion dives into the intriguing world of p-hacking and its many aliases like data dredging. The hosts explore how flexible analytics can lead to inflated error rates and why terminology matters in research ethics. They debate the fine line between questionable practices and outright fraud, analyzing the personal sensitivities surrounding such accusations. Listeners will learn about common p-hacking strategies, the implications of analysis choices, and the challenge researchers face in clarifying past behaviors. Tune in for a thought-provoking journey!
AI Snips
Chapters
Books
Transcript
Episode notes
P-Hacking Defined By Inflated Error Rates
- P-hacking describes making analysis decisions after seeing data that selectively increase publishable results.
- The core harm is inflating error rates, not just pushing p below 0.05.
Many Names, One Problem
- Many terms (data dredging, specification searching, garden of forking paths) describe similar behaviors with different imagery.
- These labels vary in connotation and clarity but point to the same broad problem of analytic flexibility.
Flexibility Isn't The Sin—Uncontrolled Error Is
- The critical distinction is whether analytic choices inflate error rates, not flexibility per se.
- If you control your error rates statistically, many flexible practices become acceptable.
