Artificial General Intelligence (AGI) Show with Soroush Pour

Ep 11 - Technical alignment overview w/ Thomas Larsen (Director of Strategy, Center for AI Policy)

16 snips

Dec 14, 2023

In this episode, Soroush Pour interviews Thomas Larsen, Director for Strategy at the Center for AI Policy. They discuss various topics including technical alignment areas such as scalable oversight, interpretability, heuristic arguments, model evaluations, agent foundations, and more. They also explore the concept of AIXI, uncomputability, building a multi-level world model, inverse reinforcement learning, and cooperative AI. The conversation concludes with a discussion on future challenges and cooperation in AI systems.

Ask episode

Chapters

Transcript

Episode notes

Introduction

00:00 • 2min

Thomas Larsen's Background and Work in AI Safety

02:02 • 9min

Interpreting AI and Model Evaluation

11:26 • 19min

Interpretability in Language Models and Machine Learning

30:22 • 10min

Exploring Interpretability and the Colin Burns Activation Probes Technique

40:45 • 2min

Critiques and Heuristic Arguments in AI Alignment Research

42:55 • 14min

Exploring AIXI and Hypercomputers in AGI

57:13 • 2min

Uncomputability and the Halting Problem

58:54 • 9min

Building a Multi-Level World Model

01:07:46 • 11min

Building AI Systems, Inverse Reinforcement Learning, and Cooperative AI

01:18:48 • 2min

Future Challenges and Cooperation in AI Systems

01:21:01 • 14min

Discussion on AI research funding and recommendations

01:35:20 • 2min

We speak with Thomas Larsen, Director for Strategy at the Center for AI Policy in Washington, DC, to do a "speed run" overview of all the major technical research directions in AI alignment. A great way to quickly learn broadly about the field of technical AI alignment.

In 2022, Thomas spent ~75 hours putting together an overview of what everyone in technical alignment was doing. Since then, he's continued to be deeply engaged in AI safety. We talk to Thomas to share an updated overview to help listeners quickly understand the technical alignment research landscape.

We talk to Thomas about a huge breadth of technical alignment areas including:

* Prosaic alignment
* Scalable oversight (e.g. RLHF, debate, IDA)
* Intrepretability
* Heuristic arguments, from ARC
* Model evaluations
* Agent foundations
* Other areas more briefly:
* Model splintering
* Out-of-distribution (OOD) detection
* Low impact measures
* Threat modelling
* Scaling laws
* Brain-like AI safety
* Inverse reinforcement learning (RL)
* Cooperative AI
* Adversarial training
* Truthful AI
* Brain-machine interfaces (Neuralink)

Hosted by Soroush Pour. Follow me for more AGI content:
Twitter: https://twitter.com/soroushjp
LinkedIn: https://www.linkedin.com/in/soroushjp/

== Show links ==

-- About Thomas --

Thomas studied Computer Science & Mathematics at U. Michigan where he first did ML research in the field of computer vision. After graduating, he completed the MATS AI safety research scholar program before doing a stint at MIRI as a Technical AI Safety Researcher. Earlier this year, he moved his work into AI policy by co-founding the Center for AI Policy, a nonprofit, nonpartisan organisation focused on getting the US government to adopt policies that would mitigate national security risks from AI. The Center for AI Policy is not connected to foreign governments or commercial AI developers and is instead committed to the public interest.

* Center for AI Policy - https://www.aipolicy.us
* LinkedIn - https://www.linkedin.com/in/thomas-larsen/
* LessWrong - https://www.lesswrong.com/users/thomas-larsen

-- Further resources --

* Thomas' post, "What Everyone in Technical Alignment is Doing and Why" https://www.lesswrong.com/posts/QBAjndPuFbhEXKcCr/my-understanding-of-what-everyone-in-technical-alignment-is
* Please note this post is from Aug 2022. The podcast should be more up-to-date, but this post is still a valuable and relevant resource.