LessWrong (Curated & Popular) "Good if make prior after data instead of before" by dynomight
Dec 27, 2025
The discussion dives into the concept of setting priors before analyzing data, exploring why this traditional approach might not always work. Using aliens as a thought-provoking example, the host reveals how ambiguous evidence complicates belief updating. Different types of aliens illustrate the necessity of finer categorization to reach more accurate conclusions. The conversation emphasizes the pitfalls of rigid prior assumptions and advocates for a data-informed approach to hypothesis formation. Ultimately, listeners are encouraged to rethink how they assess likelihoods.
AI Snips
Chapters
Transcript
Episode notes
Priors Alone Don't Solve Complex Problems
- Bayes' rule assumes priors encode all pre-data information and updates via likelihoods.
- In practice, refusing to revise priors after seeing data can cause mistakes when the hypothesis space is complex.
Aliens Example Reveals Bayesian Oddities
- Dynamite uses the alien-on-Earth example to show Bayesian intuition can give absurd results.
- He contrasts equal prior plausibility with messy real-world evidence like pilot reports and the WOW! Signal.
Hypotheses Hide Many Distinct Cases
- The hypothesis 'there are aliens' actually decomposes into many distinct possibilities with different predictive patterns.
- Coarse priors that ignore these subtypes can mask important differences in likelihoods and produce misleading posteriors.
![Diagram showing probability brackets P[good] in red and P[bad] in blue.](https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/JAA2cLFH7rLGNCeCo/gsq18mbjzljqayglbzrz)
