AXRP - the AI X-risk Research Podcast 38.0 - Zhijing Jin on LLMs, Causality, and Multi-Agent Systems
4 snips
Nov 14, 2024 Zhijing Jin, an Assistant Professor at the University of Toronto, specializes in the intersection of natural language processing and causal inference. In this engaging discussion, she investigates whether language models truly understand causality or just recognize correlations. Zhijing explores the limitations of these models in reasoning, their application in multi-agent systems, and the complexities of digital societies. She poses intriguing questions about AI governance and cooperation, emphasizing the delicate balance required for sustainable agent interactions.
AI Snips
Chapters
Transcript
Episode notes
Causality's Role in AI Alignment
- Causal inference helps AI alignment by linking AI's actions to their consequences.
- Understanding causal effects is foundational before adding moral or reward-based decision layers.
Applying Causality to Deterministic Models
- In deterministic LLMs, causal inference techniques apply by intervention on neurons to observe effects.
- Methods include ablating neurons, mediation analysis, and causal abstraction at a macro level.
Understanding Causal Abstraction Advances
- Causal abstraction maps neurons to higher-level functions to understand model computations.
- Recent advances include bottom-up approaches like Sparse Autoencoders to reduce neuron space dimensionality.