AI Safety Fundamentals: Alignment cover image

What Failure Looks Like

AI Safety Fundamentals: Alignment

CHAPTER

The Problem With Influence Seeking Policies in Machine Learning

Modern ML instantiates massive numbers of cognitive policies, then further refines and ultimately deploys whatever policies perform well according to some training objective. If progress continues, eventually machine learning will probably produce systems that have a detailed understanding of the world which are able to adapt their behavior in order to achieve specific goals. Any influence seeking policies we stumble across would also score well according to our training objective, because performing well on the training objective is a good strategy for obtaining influence. How frequently will we run into influence seeking policies versus policies that just straightforwardly pursue the goals we want of them to? I don't know. One reason to be scared is that a wide variety of goals could lead to influence seeking

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner