Science Fictions Episode 23: Statistical significance
42 snips
Jan 9, 2024 The podcast discusses statistical significance and p-values, exploring misconceptions and ways to prevent p-hacking. It also covers the problem of false positives, the controversy around power posing, and the link between reading on an e-reader and delayed sleep onset. The hosts highlight the importance of interpreting p-values correctly and consider potential solutions such as pre-registration and Bayesian statistics.
AI Snips
Chapters
Transcript
Episode notes
Statistics Detect Tendencies, Not Certainties
- Statistical tests detect tendencies across samples, not absolute truths about populations.
- Sampling error and bias make observed differences ambiguous without careful inference.
The Literary Digest Poll Failure
- The 1936 Literary Digest poll sampled millions yet predicted the wrong winner due to biased sampling frames.
- George Gallup used a smaller but more representative sample and correctly predicted Roosevelt's victory.
What A P-Value Actually Measures
- A p-value is the probability of observing the data (or more extreme) assuming the null hypothesis is true.
- It measures surprisingness under the null, not the probability the null is true.