Machine Learning Street Talk (MLST) AI Alignment & AGI Fire Alarm - Connor Leahy
5 snips
Nov 1, 2020 Connor Leahy, a machine learning engineer from Aleph Alpha and founder of EleutherAI, dives into the urgent complexities of AI alignment and AGI. He argues that AI alignment is philosophy with a deadline, likening AGI's challenges to climate change but with even more catastrophic potential. The discussion touches on decision theories like Newcomb's paradox, the prisoner's dilemma, and the dangers of poorly defined utility functions. Together, they unravel the philosophical implications of AI, the nature of intelligence, and the dire need for responsible action in AI development.
AI Snips
Chapters
Books
Transcript
Episode notes
Stop Button Problem
- The stop button problem highlights the difficulty of designing AI that willingly shuts down.
- An AI might prioritize an easy reward for self-suspension over a complex task, even if it means not completing the intended objective.
Google as AI
- The concept of shutting down an advanced AI, like Google, becomes increasingly difficult and undesirable.
- Intelligence externalized in complex systems becomes nebulous, diffuse, and self-healing, resistant to traditional off switches.
Rationality and Dutch Booking
- Define rationality as immunity to Dutch booking, where one consistently loses resources due to flawed decision-making.
- Ensure your decision-making theory prevents infinite resource extraction for no reason, addressing potential vulnerabilities.
