LessWrong (Curated & Popular)

“Self-Other Overlap: A Neglected Approach to AI Alignment” by Marc Carauleanu, Mike Vaiana, Judd Rosenblatt, Diogo de Lucena

Aug 7, 2024

Join guests Bogdan Ionut-Cirstea, Steve Byrnes, Gunnar Zarnacke, Jack Foxabbott, and Seong Hah Cho, who contribute critical insights on AI alignment. They discuss an intriguing concept called self-other overlap, which aims to optimize AI models by aligning their reasoning about themselves and others. Early experiments suggest this technique can reduce deceptive behaviors in AI. With its scalable nature and minimal need for interpretability, self-other overlap could be a game-changer in creating pro-social AI.

23:21

Episode guests

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Self-other overlap training aims to align AI models with human values by minimizing distinctions between self and others, reducing deceptive behavior.

Early experiments show that AI agents trained with self-other overlap demonstrate significantly less deception and align more closely with non-deceptive behavior.

Deep dives

Introduction to Self-Other Overlap Training

Self-other overlap training focuses on creating similar internal representations when an AI model reasons about itself and others. This technique aims to reduce deceptive behavior by minimizing the distinctions the model makes between itself and external agents, ultimately leading to alignment with human values. Evidence suggests that neural self-other overlap in humans relates to prosociality, thus proposing its potential relevance for AI alignment. By optimizing for self-other overlap, the approach requires minimal interpretability, making it scalable and adaptable for various models with little to no disruptive effects on their capabilities.

Exploring Self-Other Overlap for AI Alignment

23min

Figure 1. Image generated by DALL-3 to represent the concept of self-other overlapMany thanks to Bogdan Ionut-Cirstea, Steve Byrnes, Gunnar Zarnacke, Jack Foxabbott and Seong Hah Cho for critical comments and feedback on earlier and ongoing versions of this work.

Summary

In this post, we introduce self-other overlap training: optimizing for similar internal representations when the model reasons about itself and others while preserving performance. There is a large body of evidence suggesting that neural self-other overlap is connected to pro-sociality in humans and we argue that there are more fundamental reasons to believe this prior is relevant for AI Alignment. We argue that self-other overlap is a scalable and general alignment technique that requires little interpretability and has low capabilities externalities. We also share an early experiment of how fine-tuning a deceptive policy with self-other overlap reduces deceptive behavior in a simple RL environment. On top of that [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:
July 30th, 2024

Source:
https://www.lesswrong.com/posts/hzt9gHpNwA2oHtwKX/self-other-overlap-a-neglected-approach-to-ai-alignment

---

Narrated by TYPE III AUDIO.

---

Images from the article: