The Risk of Overfitting in the Brain

In this bonus episode, Peter Bandettini talks to four co-authors from a recent Nature paper on “Reproducible brain-wide association studies require thousands of individuals.” Scott Marek, Brenden Tervo-Clemmens, Damien Fair and Nico Dosenbach discuss their work, demonstrating that to make reproducible associations between MRI measures (both structural and functional) and behavioral measures, upwards of 2000 subjects are required.

The panel discuss the strong reaction across the field to this paper, and how the results fit with the known strong and robust signal from fMRI. They consider why the effect size is essentially three orders of magnitude smaller when trying to pull out differences between subjects. In this insightful, clarifying, and ultimately optimistic conversation about fMRI and the implications of this paper, Peter and his guests go over possible reasons for these extremely small effects, and discuss ways forward.