Understanding AI Alignment: The Preferentist Approach

This chapter delves into the concept of alignment in artificial intelligence, emphasizing the preferentist approach to harmonizing AI systems with human preferences. It discusses the role of human feedback in training AI and addresses the challenges faced in achieving effective alignment.

Transcript

chevron_right

Play full episode

chevron_right

Transcript

Episode notes

In this episode of The Cognitive Revolution, Nathan explores groundbreaking perspectives on AI alignment with MIT PhD student Tan Zhi Xuan. We dive deep into Xuan's critique of preference-based AI alignment and their innovative proposal for role-based AI systems guided by social consensus. The conversation extends into their fascinating work on how AI agents can learn social norms through Bayesian rule induction. Join us for an intellectually stimulating discussion that bridges philosophical theory with practical implementation in AI development.

Check out:

"Beyond Preferences in AI Alignment" paper: https://arxiv.org/pdf/2408.16984

"Learning and Sustaining Shared Normative Systems via Bayesian Rule Induction in Markov Games" paper: https://arxiv.org/pdf/2402.13399

Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse

SPONSORS:

Notion: Notion offers powerful workflow and automation templates, perfect for streamlining processes and laying the groundwork for AI-driven automation. With Notion AI, you can search across thousands of documents from various platforms, generating highly relevant analysis and content tailored just for you - try it for free at https://notion.com/cognitiverevolution

Weights & Biases RAG++: Advanced training for building production-ready RAG applications. Learn from experts to overcome LLM challenges, evaluate systematically, and integrate advanced features. Includes free Cohere credits. Visit https://wandb.me/cr to start the RAG++ course today.

Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers13. OCI powers industry leaders with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before December 31, 2024 at https://oracle.com/cognitive

RECOMMENDED PODCAST:

Unpack Pricing - Dive into the dark arts of SaaS pricing with Metronome CEO Scott Woody and tech leaders. Learn how strategic pricing drives explosive revenue growth in today's biggest companies like Snowflake, Cockroach Labs, Dropbox and more.

Apple: https://podcasts.apple.com/us/podcast/id1765716600

Spotify: https://open.spotify.com/show/38DK3W1Fq1xxQalhDSueFg

CHAPTERS:

(00:00:00) Teaser

(00:01:09) About the Episode

(00:04:25) Guest Intro

(00:06:25) Xuan's Background

(00:12:03) AI Near-Term Outlook

(00:17:32) Sponsors: Notion | Weights & Biases RAG++

(00:20:18) Alignment Approaches

(00:26:11) Critiques of RLHF

(00:34:40) Sponsors: Oracle Cloud Infrastructure (OCI)