AXRP - the AI X-risk Research Podcast cover image

22 - Shard Theory with Quintin Pope

AXRP - the AI X-risk Research Podcast

CHAPTER

The Implications of Char-D Theory for Deep Learning

In char-d theory there's this account of how these shards form which is like you know in certain you happen to be doing a thing and that gets you some reward and that forms a shard which says yeah do that thing in this context right. Once you've formed these shards you're now going to do your own optimization not just for what your reward circuitry wanted but for the goals you've learned during training. So alignment is going to be very difficult because humans aren't optimizing for their reward circuitry they're not even optimizing that hard for learned optimization functions so  it's kind of a mess and it's very hard to steer.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner