Science Fictions A Christmas 2025 compendium
10 snips
Dec 30, 2025 Stuart and Tom dive into the world of questionable science, detailing a rise in low-quality AI-generated papers and letters to journals. They tackle the pitfalls of automated data-mining and the 'crud' effect, where large datasets yield trivial correlations. The duo discusses the problematic use of AI in peer review but also highlights its potential for error detection. They wrap up with a compelling exposé on the Dana-Farber scandal, revealing image manipulation and the subsequent legal settlement that rewards whistleblowing. A mix of caution and optimism fills the conversation!
AI Snips
Chapters
Books
Transcript
Episode notes
AI Reveals Correlations — Not Always Truth
- AI can massively speed hypothesis generation by scanning public datasets for correlations.
- But automated, unprincipled scanning risks huge false positives and worthless papers.
Big Data Guarantees Spurious Links
- Large sample sizes and unlimited variable searches guarantee trivial correlations.
- That 'crud' effect means many AI-found links will be real but meaningless.
Require Replication And Independent Splits
- When using public datasets, split data into independent samples and replicate findings externally.
- Treat AI-generated hypotheses as starting points, not definitive results.
