80,000 Hours Podcast
#212 – Allan Dafoe on why technology is unstoppable & how to shape AI development anyway
Episode guests
Podcast summary created with Snipd AI
Quick takeaways
- Technology provides opportunities rather than compulsion, influencing societies through military and economic competitiveness to adopt transformative capabilities.
- Allan Dafoe emphasizes the importance of governance frameworks and collaborative efforts to anticipate risks associated with emerging AI technologies.
- The integration of Google DeepMind allows for multidisciplinary collaboration to address regulatory and safety norms surrounding advanced AI deployment.
- Proactive governance is essential in mitigating risks from rapidly advancing technologies, focusing on advancing beneficial innovations ahead of harmful ones.
- Cooperative AI aims to enhance interactions among AI systems and between AI and humans, promoting safer outcomes in autonomous decision-making.
Deep dives
The Role of Technology in Shaping Choices
Technology does not compel individuals to act in specific ways; rather, it presents opportunities for new forms of living and behavior. The argument suggests that while many groups might choose to ignore emerging technologies, those who adopt them often gain significant advantages. This competitive pressure can lead to a situation where others are forced to adapt to avoid falling behind. Thus, the dynamics of military and economic competition often dictate the choices societies ultimately make regarding technological adoption.
Alan Defoe's Expertise and Role
Alan Defoe serves as the Director of Frontier Safety and Governance at Google DeepMind, where he oversees three main pillars: frontier safety, governance, and planning. His team evaluates potential risks associated with emerging technologies, advising on necessary safety frameworks and regulations. The goal is to anticipate new issues that might arise with advancements in AI and ensure safe and responsible development. Collaborative efforts with various technical and policy teams facilitate a comprehensive approach to addressing these challenges.
The Global AI Landscape and Integration
Google DeepMind specializes in frontier models, focusing on the safety and governance of these technologies while closely collaborating with other teams within Google's broader structure. The integration of DeepMind into Google's ecosystem has fostered multi-disciplinary collaboration, enhancing the discourse on frontier issues. This collaboration aims to identify and address the policies, regulations, and safety norms that must accompany the deployment of advanced AI technologies. Another critical aspect is staying ahead of potential threats by continuously evaluating the capabilities of emerging models.
From Governance of AI to Practical Implementation
Defoe transitioned from leading the Governance of AI Center to his current role at Google DeepMind to increase his impact on AGI safety and governance. His experience in advising influential decision-makers positions him to better integrate safety considerations into real-world applications of AI. Understanding the profound implications of AI development, he emphasizes the importance of having capable leaders who are safety-oriented and aware of the stakes involved. This focus on character and competence in decision-making is crucial for addressing the complexities of AI governance.
Technological Determinism and Its Implications
The conversation around technological determinism highlights the intricate interplay between technology and historical development. While historical and macro-level trends influence societal outcomes, the role of human agency in steering technology continues to be debated. Some argue that societal structures and power dynamics shape technological trajectories, while others maintain the view that human decisions dictate outcomes. Understanding this relationship is vital for developing informed policies around the adoption and governance of new technologies.
Differential Technological Development and Its Risks
Differential technological development emphasizes the need to advance beneficial technologies ahead of harmful ones to minimize risks. This approach includes developing safety measures and incentives for positive outcomes before harmful technologies become prevalent. However, challenges arise from predicting which technologies will gain traction and be adopted. A cautionary perspective suggests that rapid advancements can lead to unforeseen consequences, necessitating proactive governance frameworks to mitigate risks.
Cooperative AI as a Crucial Research Focus
Cooperative AI seeks to enhance collaborative skills in artificial intelligence systems, fostering better interactions among them. The idea is to invest in developing systems that can work together effectively, leading to safer outcomes and reducing risks associated with autonomous decision-making. This concept extends to improving interactions between AI systems and humans, aiming to create a more harmonious coexistence. By prioritizing cooperative capabilities, researchers hope to address potential challenges posed by powerful AI technologies.
The Future of AI in Education
AI in education has the potential to revolutionize learning experiences by providing personalized and adaptive1-tutoring. By catering to individual needs, AI can help students who struggle or push advanced students to their full potential. This technology may also alleviate some burdens on traditional education systems, offering additional support where it's most needed. As AI systems become more integrated into classrooms, their role as educational assistants could lead to enhanced engagement and improved outcomes for students.
AI's Role in Healthcare and Medicine
AI technologies like deep learning and LLMs hold the potential to transform healthcare by streamlining processes, supporting diagnosis, and increasing accessibility to expert knowledge. Innovative tools could assist doctors in patient consultations, manage treatment protocols, and expedite drug discovery through computational research. This increased efficiency and capability may lead to greater early detection of diseases and more personalized medicine. The ongoing developments in AI for health and medicine signal a promising future for better patient care.
Addressing Governance Through Collective Efforts
Governance around frontier AI technologies requires collaboration across various disciplines, involving contributions from policymakers, researchers, and industry professionals. Initiatives like the Frontier Model Forum bring together key stakeholders to establish safety standards and shared governance structures. Through active engagement, these collaborative efforts aim to balance innovation and safety, ultimately leading to responsible development of AI technologies. Ensuring that societal concerns are addressed requires a concerted effort across diverse sectors.
Technology doesn’t force us to do anything — it merely opens doors. But military and economic competition pushes us through.
That’s how today’s guest Allan Dafoe — director of frontier safety and governance at Google DeepMind — explains one of the deepest patterns in technological history: once a powerful new capability becomes available, societies that adopt it tend to outcompete those that don’t. Those who resist too much can find themselves taken over or rendered irrelevant.
Links to learn more, highlights, video, and full transcript.
This dynamic played out dramatically in 1853 when US Commodore Perry sailed into Tokyo Bay with steam-powered warships that seemed magical to the Japanese, who had spent centuries deliberately limiting their technological development. With far greater military power, the US was able to force Japan to open itself to trade. Within 15 years, Japan had undergone the Meiji Restoration and transformed itself in a desperate scramble to catch up.
Today we see hints of similar pressure around artificial intelligence. Even companies, countries, and researchers deeply concerned about where AI could take us feel compelled to push ahead — worried that if they don’t, less careful actors will develop transformative AI capabilities at around the same time anyway.
But Allan argues this technological determinism isn’t absolute. While broad patterns may be inevitable, history shows we do have some ability to steer how technologies are developed, by who, and what they’re used for first.
As part of that approach, Allan has been promoting efforts to make AI more capable of sophisticated cooperation, and improving the tests Google uses to measure how well its models could do things like mislead people, hack and take control of their own servers, or spread autonomously in the wild.
As of mid-2024 they didn’t seem dangerous at all, but we’ve learned that our ability to measure these capabilities is good, but imperfect. If we don’t find the right way to ‘elicit’ an ability we can miss that it’s there.
Subsequent research from Anthropic and Redwood Research suggests there’s even a risk that future models may play dumb to avoid their goals being altered.
That has led DeepMind to a “defence in depth” approach: carefully staged deployment starting with internal testing, then trusted external testers, then limited release, then watching how models are used in the real world. By not releasing model weights, DeepMind is able to back up and add additional safeguards if experience shows they’re necessary.
But with much more powerful and general models on the way, individual company policies won’t be sufficient by themselves. Drawing on his academic research into how societies handle transformative technologies, Allan argues we need coordinated international governance that balances safety with our desire to get the massive potential benefits of AI in areas like healthcare and education as quickly as possible.
Host Rob and Allan also cover:
- The most exciting beneficial applications of AI
- Whether and how we can influence the development of technology
- What DeepMind is doing to evaluate and mitigate risks from frontier AI systems
- Why cooperative AI may be as important as aligned AI
- The role of democratic input in AI governance
- What kinds of experts are most needed in AI safety and governance
- And much more
Chapters:
- Cold open (00:00:00)
- Who's Allan Dafoe? (00:00:48)
- Allan's role at DeepMind (00:01:27)
- Why join DeepMind over everyone else? (00:04:27)
- Do humans control technological change? (00:09:17)
- Arguments for technological determinism (00:20:24)
- The synthesis of agency with tech determinism (00:26:29)
- Competition took away Japan's choice (00:37:13)
- Can speeding up one tech redirect history? (00:42:09)
- Structural pushback against alignment efforts (00:47:55)
- Do AIs need to be 'cooperatively skilled'? (00:52:25)
- How AI could boost cooperation between people and states (01:01:59)
- The super-cooperative AGI hypothesis and backdoor risks (01:06:58)
- Aren’t today’s models already very cooperative? (01:13:22)
- How would we make AIs cooperative anyway? (01:16:22)
- Ways making AI more cooperative could backfire (01:22:24)
- AGI is an essential idea we should define well (01:30:16)
- It matters what AGI learns first vs last (01:41:01)
- How Google tests for dangerous capabilities (01:45:39)
- Evals 'in the wild' (01:57:46)
- What to do given no single approach works that well (02:01:44)
- We don't, but could, forecast AI capabilities (02:05:34)
- DeepMind's strategy for ensuring its frontier models don't cause harm (02:11:25)
- How 'structural risks' can force everyone into a worse world (02:15:01)
- Is AI being built democratically? Should it? (02:19:35)
- How much do AI companies really want external regulation? (02:24:34)
- Social science can contribute a lot here (02:33:21)
- How AI could make life way better: self-driving cars, medicine, education, and sustainability (02:35:55)
Video editing: Simon Monsour
Audio engineering: Ben Cordell, Milo McGuire, Simon Monsour, and Dominic Armstrong
Camera operator: Jeremy Chevillotte
Transcriptions: Katy Moore
Remember Everything You Learn from Podcasts
