AXRP - the AI X-risk Research Podcast cover image

AXRP - the AI X-risk Research Podcast

18 - Concept Extrapolation with Stuart Armstrong

Sep 3, 2022
01:46:19

Concept extrapolation is the idea of taking concepts an AI has about the world - say, "mass" or "does this picture contain a hot dog" - and extending them sensibly to situations where things are different - like learning that the world works via special relativity, or seeing a picture of a novel sausage-bread combination. For a while, Stuart Armstrong has been thinking about concept extrapolation and how it relates to AI alignment. In this episode, we discuss where his thoughts are at on this topic, what the relationship to AI alignment is, and what the open questions are.

 

Topics we discuss, and timestamps:

 - 00:00:44 - What is concept extrapolation

 - 00:15:25 - When is concept extrapolation possible

 - 00:30:44 - A toy formalism

 - 00:37:25 - Uniqueness of extrapolations

 - 00:48:34 - Unity of concept extrapolation methods

 - 00:53:25 - Concept extrapolation and corrigibility

 - 00:59:51 - Is concept extrapolation possible?

 - 01:37:05 - Misunderstandings of Stuart's approach

 - 01:44:13 - Following Stuart's work

 

The transcript: axrp.net/episode/2022/09/03/episode-18-concept-extrapolation-stuart-armstrong.html

 

Stuart's startup, Aligned AI: aligned-ai.com

 

Research we discuss:

 - The Concept Extrapolation sequence: alignmentforum.org/s/u9uawicHx7Ng7vwxA

 - The HappyFaces benchmark: github.com/alignedai/HappyFaces

 - Goal Misgeneralization in Deep Reinforcement Learning: arxiv.org/abs/2105.14111

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner