LessWrong (Curated & Popular) “What Is The Alignment Problem?” by johnswentworth
Jan 17, 2025
The podcast dives into the complexities of aligning future AGIs with human values. It explores illustrative toy problems to highlight the challenges in categorizing and specifying goals. The discussion emphasizes how nuanced understanding of human values is critical for effective alignment. It further examines the distinction between basic agents and general intelligence, shedding light on the difficulties in ensuring AI operates harmoniously in various environments. The conversation also touches on corrigibility and the intricacies of what alignment truly means.
AI Snips
Chapters
Transcript
Episode notes
Old McDonald's Hen
- Old McDonald wants his newest hen to be third in the pecking order.
- This illustrates how problem specification depends on environmental patterns, like a linear pecking order.
Sorting Blegs and Rubes
- A factory worker sorts 'blegs' and 'rubes' based on their properties.
- This highlights how clustering patterns are crucial for defining a sorting task.
Problem Specification and Alignment
- Goals require specific patterns to exist; defining those patterns is key to problem specification.
- Alignment relies on understanding patterns in AGI and human cognition, which are currently poorly understood.
