Effective Altruism: An Introduction – 80,000 Hours (April 2021) Seven: Prof Tetlock on why accurate forecasting matters for everything, and how you can do it better
Why Experts Often Miss The Mark
- Experts are often overconfident and only slightly better than random at forecasting complex events.
- The best forecasters combine curiosity, open-mindedness and tolerance for dissonance.
Superforecasters Outperform Institutions
- A small subset of people — superforecasters — consistently produce much better probabilistic predictions.
- Aggregating judgments from many good forecasters often beats official intelligence estimates.
Why Use Simulations For Counterfactuals
- Counterfactual reasoning in real history is ambiguous and often ideologically self-serving.
- Simulated worlds (e.g., Civ V) let researchers test counterfactual forecasting empirically.
Have you ever been infuriated by a doctor's unwillingness to give you an honest, probabilistic estimate about what to expect? Or a lawyer who won't tell you the chances you'll win your case?
Their behaviour is so frustrating because accurately predicting the future is central to every action we take. If we can't assess the likelihood of different outcomes we're in a complete bind, whether the decision concerns war and peace, work and study, or Black Mirror and RuPaul's Drag Race.
Which is why the research of Professor Philip Tetlock is relevant for all of us each and every day.
In this conversation from 2019, we discuss how his work can be applied to your personal life to answer high-stakes questions, like how likely you are to thrive in a given career path, or whether your business idea will be a billion-dollar unicorn — or fall apart catastrophically.
This episode first broadcast on the regular 80,000 Hours Podcast feed on June 28, 2019. Some related episodes include:
• #7 – Julia Galef on making humanity more rational, what EA does wrong, and why Twitter isn’t all bad
• #11 – Dr Spencer Greenberg on speeding up social science 10-fold & why plenty of startups cause harm.
• #15 – Prof Tetlock on how chimps beat Berkeley undergrads and when it’s wise to defer to the wise
• #30 – Dr Eva Vivalt on how little social science findings generalize from one study to another
• #40 – Katja Grace on forecasting future technology & how much we should trust expert predictions.
• #48 – Brian Christian on better living through the wisdom of computer science
• #78 – Danny Hernandez on forecasting and measuring some of the most important drivers of AI progress
Series produced by Keiran Harris.