LessWrong (30+ Karma) “The Moonrise Problem” by johnswentworth
Nov 30, 2025
Explore the intriguing Moonrise incident where a false alarm about Soviet missiles was triggered by lunar reflection. Delve into the challenges of program verification in high-stakes systems, questioning whether it's truly possible to predict unexpected events. Discover how machine learning models can address these issues, albeit with limitations on understanding complex human concepts. The conversation highlights the importance of robust assumptions in developing systems that can handle real-world unpredictability.
AI Snips
Chapters
Books
Transcript
Episode notes
Thule False Alarm Due To Moonrise
- On October 5, 1960, Thule's radar falsely indicated a Soviet missile attack because the moon reflected radar signals back to Earth.
- Engineers hadn't predicted lunar reflection, so common sense prevented a catastrophic counterattack.
Verification Proves Specs, Not Reality
- Program verification can only prove a program matches its specification, not that the spec matches reality.
- If engineers miss a real-world phenomenon when writing the spec, verification won't prevent failures like the moonrise case.
Moonrise Problem Is Solvable In Principle
- Wentworth argues the moonrise problem is solvable in principle without exhaustive prior knowledge of every complication.
- He claims one could prove robust correctness of a warning system so it handles novel, unforeseen situations with high probability.
