The Studies Show

Episode 66: Superforecasting

Mar 4, 2025

Explore the intriguing world of superforecasting, where accurate predictions come from understanding probabilistic language. Dive into Cold War tensions that shaped critical decision-making moments and learn why teamwork may not always enhance forecasting accuracy. Discover the nuanced skills that distinguish exceptional forecasters, while reflecting on the dangers of flawed research in Alzheimer's studies. Embrace open-mindedness and the power of Bayesian reasoning to refine beliefs and improve predictive capabilities.

01:15:53

Creator website

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Superforecasting leverages statistical approaches and numerical probabilities to enhance predictive accuracy and facilitate continuous learning in predictions.

Historical examples illustrate the severe consequences of poor forecasting in high-stakes situations, emphasizing the need for reliable predictors to ensure national security.

Experts often fail at accurate predictions compared to superforecasters, highlighting the necessity of structured prediction methods for improved decision-making.

Deep dives

The Art of Superforecasting

Superforecasting is a technique that involves predicting future events with a focus on accuracy and calibration. It relies on a statistical approach where forecasters assign numeric probabilities to their predictions, rather than vague terms. This method helps in evaluating the accuracy of forecasts over time, enabling forecasters to improve their predictions based on past performance. The process emphasizes continuous learning and adaptation, allowing individuals to refine their predictive abilities.

Intro

2min

Cold War Tensions and Predictions

11min

Understanding Probabilistic Language and Forecasting

10min

The Art of Superforecasting

13min

Unmasking Flawed Research

4min

Exploring the Art of Forecasting and Improvement

2min

The Psychology of Forecasting

14min

Embracing Open-Mindedness and the Outside View

2min

Forecasting Success: Marriages and Divorce Rates

4min

10.

Bayesian Reasoning in Superforecasting

15min

Whether it’s the 1903 New York Times article that claimed a flying machine was ten million years away, or the record executive who (allegedly) told the Beatles in the early 1960s that guitar bands were on the way out, predictions are hard.

In this episode of The Studies Show, Tom and Stuart discuss the psychologist Philip Tetlock’s research on superforecasters, the people who make the most accurate predictions of all. Even if you can’t become a superforecaster yourself, it turns out there’s a lot we can learn from them about how to form beliefs—and how to be right more often.

The Studies Show is brought to you by Works in Progress magazine, where this week Tom has written a review of the new book, Doctored, about fraud in Alzheimer’s research. Read that and many other short pieces on the Works in Progress Substack at worksinprogress.news.

Show notes

* A book chapter on the “Expert Political Judgement” study from Philip Tetlock

* Research on how people interpret terms like “a serious possibility” and “likely”

* Research that argues against the idea that teaming up makes superforecasters better

* Study on the correlates of being a good superforecaster (i.e. having a low Brier score)

* A paper on “small steps to accuracy”: how people who update their beliefs more often are better forecasters

* Philip Tetlock and Dan Gardner’s book Superforecasting

* Julia Galef’s book The Scout Mindset

* Tom’s book, Everything is Predictable

* Tom’s review of Mervyn King’s book, Radical Uncertainty

Credits

The Studies Show is produced by Julian Mayers at Yada Yada Productions.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.thestudiesshowpod.com/subscribe

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.

The Studies Show

Episode 66: Superforecasting

Podcast summary created with Snipd AI

Quick takeaways

Deep dives

The Art of Superforecasting

Historical Context of Predictions

The Power of Base Rates

Challenges with Expert Predictions

Practical Applications of Forecasting

Remember Everything You Learn from Podcasts