AI Alignment & AGI Fire Alarm - Connor Leahy

5 snips

Nov 1, 2020

Connor Leahy, a machine learning engineer from Aleph Alpha and founder of EleutherAI, dives into the urgent complexities of AI alignment and AGI. He argues that AI alignment is philosophy with a deadline, likening AGI's challenges to climate change but with even more catastrophic potential. The discussion touches on decision theories like Newcomb's paradox, the prisoner's dilemma, and the dangers of poorly defined utility functions. Together, they unravel the philosophical implications of AI, the nature of intelligence, and the dire need for responsible action in AI development.

Ask episode

AI Snips

Chapters

Books

Transcript

Episode notes

INSIGHT

AI Alignment Schools of Thought

AI alignment research has diverse approaches due to its colorful history, originating from transhumanist newsletters.
This field has recently gained mainstream recognition, leading to varying levels of familiarity with older, more radical views.

INSIGHT

AI Alignment Approaches

AI alignment approaches vary in problem difficulty, timeline, and solution requirements, ranging from prosaic to radical.
Prosaic AI alignment assumes future AI will resemble current AI, focusing on aligning neural networks, while MIRI emphasizes fundamental understanding.

INSIGHT

Defining Intelligence

Connor Leahy defines intelligence practically as the ability to solve problems, prioritizing practical usefulness over philosophical definitions.
He emphasizes focusing on optimization processes and pressure, measuring a system's power by its ability to influence values.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Stubborn Attachments

Tyler Cowen

Atlas Shrugged

Ayn Rand

Published in 1957, 'Atlas Shrugged' is Ayn Rand's magnum opus and her longest novel. The story is set in a dystopian United States where increasingly burdensome laws and regulations strangle innovation and productivity. The plot follows Dagny Taggart, a railroad executive, and Hank Rearden, a steel magnate, as they struggle against 'looters' who exploit their work. A mysterious figure named John Galt leads a strike of productive individuals, persuading them to abandon their companies and disappear. The novel culminates with Galt's three-hour radio speech explaining his philosophy of Objectivism, which emphasizes rational self-interest, individual rights, and the importance of the human mind. The book explores themes of capitalism, property rights, and the failures of governmental coercion, presenting a provocative vision of a society in collapse and the potential for a new capitalist society based on Galt's principles.

Why greatness cannot be planned

Kenneth Stanley

The AI Does Not Hate You

Superintelligence, Rationality, and the Race to Save the World

Tom Chivers

This book provides an in-depth look at the rationalist community, focusing on their concerns about superintelligence and its potential risks to humanity. Through interviews and analysis, Chivers examines the rationalists' perspectives on AI and their efforts to mitigate potential dangers. The book offers a balanced view, exploring both the ideas and criticisms within this community.

On Intelligence

How a New Understanding of the Brain Will Lead to the Creation of Truly Intelligent Machines

Jeff Hawkins

Sandra Blakeslee

In 'On Intelligence,' Jeff Hawkins, with the help of Sandra Blakeslee, outlines his memory-prediction framework theory of the brain. This theory posits that the brain is a hierarchical, predictive system that uses memory to make continuous predictions about future events. Hawkins argues that current approaches to artificial intelligence are flawed because they do not understand the fundamental principles of how the brain works. He explains how the neocortex, the seat of intelligence, operates through a hierarchical structure, making predictions based on associative memory. The book discusses the implications of this theory for neuroscience, the development of intelligent machines, and our understanding of human behavior and cognition.