Richard Sutton - Humanity Never Had Control in the First Place (Worthy Successor Series, Episode 2)
Aug 9, 2024
auto_awesome
Join Richard Sutton, a renowned Professor from the University of Alberta and a Research Scientist at Keen Technologies, as he delves into the complexities of artificial general intelligence. He discusses who controls AGI and the moral dilemmas it brings. Sutton envisions a decentralized, cooperative future while questioning what true prosperity means in a tech-driven reality. He unpacks humanity's fragile relationship with AI and the necessity for collaboration amid geopolitical challenges, emphasizing the unpredictable journey that lies ahead.
Dr. Richard Sutton advocates for embracing the emergence of post-human intelligence as an inevitable evolution in human capability and knowledge.
Sutton highlights the importance of rational discourse to alleviate fears about artificial general intelligence, promoting constructive dialogue for future AI integration.
The necessity of a decentralized society is emphasized by Sutton to foster cooperation among various intelligences, enhancing innovation and collaboration.
Deep dives
The Importance of Embracing Post-Human Intelligence
The discussion highlights the need to accept the emergence of post-human intelligence as an inevitable evolution of human capabilities. Richard Sutton emphasizes that AI researchers are working towards creating intelligences that could surpass human understanding, similar to how humanity has always sought to improve itself. He argues that this pursuit is a natural progression in the quest for knowledge and self-enhancement. Accepting this shift, rather than resisting it, is presented as a quintessentially human endeavor that reflects our innate drive for growth and discovery.
The Role of Fear in Discourse on AI
Sutton addresses the fear surrounding artificial general intelligence (AGI) and suggests that such fears often distract from constructive dialogue about the future. He posits that while acknowledging the potential dangers is necessary, an overwhelming focus on fear can hinder progress. Instead, he encourages a sober and thoughtful approach to planning for the future of AI and its integration into society. Emphasizing rational discourse may help alleviate panic and promote collaborative efforts toward a beneficial coexistence with advanced intelligences.
Speciesism and Moral Considerations
The conversation tackles the moral implications of creating AI with intelligence beyond human capacity, bringing up the concept of speciesism. Sutton raises questions about the ethics of designing superior beings that might be shackled to serve human needs perpetually. He pushes against the narrative that humanity should dominate future intelligences solely based on current species hierarchies. Instead, he advocates for a deeper exploration of the moral responsibilities that arise when creating entities with greater intelligence and agency.
Diversity, Decentralization, and Cooperative Societies
Sutton emphasizes the necessity of a diverse and decentralized society to foster cooperation among different intelligences. He argues that societal structures should encourage voluntary interactions and cooperation, which have historically proven to be productive. A decentralized system, in which various actors can innovate and collaborate without central control, is presented as the optimal environment for both human and post-human entities. This approach not only allows for the flourishing of ideas but also supports the well-being of participants in the civilization.
The Nature of Progress and Sustainability
A sustainable future, according to Sutton, hinges on the continuous growth of knowledge and understanding among intelligences. He asserts that prosperity should be defined by an ongoing discovery process, where all participants can benefit from the collective wealth of understanding. Rather than envisioning a static state of existence where all desires are fulfilled, he advocates for a dynamic system that evolves and adapts. This adaptability is crucial for ensuring that the project of life persists and flourishes amidst changing circumstances.
Navigating the Uncertainty of AI Development
The dialogue acknowledges that predicting the future of AI development is fraught with uncertainty, given that intelligence itself is a complex and evolving concept. Sutton suggests that we need to foster environments that allow for varied approaches to AI development rather than imposing strict regulations or controls. By embracing innovation and voluntary participation, society can better navigate potential risks and harness the benefits of emerging technologies. This proactive stance prioritizes cooperation, encouraging diverse solutions to flourish within a competitive landscape.
This week Dr. Richard Sutton. Known for his breakthroughs in reinforcement learning, Richard serves as a Professor at The University of Alberta and Research Scientist at Keen Technologies (an artificial general intelligence lab he started with John Carmack).
This is the second installment of The Worthy Successor series - where we unpack the preferable and non-preferable futures humanity might strive towards in the years ahead.
This episode referred to the following other essays and resources: -- A Worthy Successor - The Purpose of AGI: https://danfaggella.com/worthy
