Understanding Preferred States and Variational Free Energy in Agents

Dr. Sanjeev Namjoshi, a machine learning engineer who recently submitted a book on Active Inference to MIT Press, discusses the theoretical foundations and practical applications of Active Inference, the Free Energy Principle (FEP), and Bayesian mechanics. He explains how these frameworks describe how biological and artificial systems maintain stability by minimizing uncertainty about their environment.

DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)?

MLST is sponsored by Tufa Labs:

Focus: ARC, LLMs, test-time-compute, active inference, system2 reasoning, and more.

Future plans: Expanding to complex environments like Warcraft 2 and Starcraft 2.

Interested? Apply for an ML research position: benjamin@tufa.ai

Namjoshi traces the evolution of these fields from early 2000s neuroscience research to current developments, highlighting how Active Inference provides a unified framework for perception and action through variational free energy minimization. He contrasts this with traditional machine learning approaches, emphasizing Active Inference's natural capacity for exploration and curiosity through epistemic value.

He sees Active Inference as being at a similar stage to deep learning in the early 2000s - poised for significant breakthroughs but requiring better tools and wider adoption. While acknowledging current computational challenges, he emphasizes Active Inference's potential advantages over reinforcement learning, particularly its principled approach to exploration and planning.

Dr. Sanjeev Namjoshi

https://snamjoshi.github.io/

TOC:

1. Theoretical Foundations: AI Agency and Sentience

[00:00:00] 1.1 Intro

[00:02:45] 1.2 Free Energy Principle and Active Inference Theory

[00:11:16] 1.3 Emergence and Self-Organization in Complex Systems

[00:19:11] 1.4 Agency and Representation in AI Systems

[00:29:59] 1.5 Bayesian Mechanics and Systems Modeling

2. Technical Framework: Active Inference and Free Energy

[00:38:37] 2.1 Generative Processes and Agent-Environment Modeling

[00:42:27] 2.2 Markov Blankets and System Boundaries

[00:44:30] 2.3 Bayesian Inference and Prior Distributions

[00:52:41] 2.4 Variational Free Energy Minimization Framework

[00:55:07] 2.5 VFE Optimization Techniques: Generalized Filtering vs DEM

3. Implementation and Optimization Methods

[00:58:25] 3.1 Information Theory and Free Energy Concepts

[01:05:25] 3.2 Surprise Minimization and Action in Active Inference

[01:15:58] 3.3 Evolution of Active Inference Models: Continuous to Discrete Approaches

[01:26:00] 3.4 Uncertainty Reduction and Control Systems in Active Inference

4. Safety and Regulatory Frameworks

[01:32:40] 4.1 Historical Evolution of Risk Management and Predictive Systems

[01:36:12] 4.2 Agency and Reality: Philosophical Perspectives on Models

[01:39:20] 4.3 Limitations of Symbolic AI and Current System Design

[01:46:40] 4.4 AI Safety Regulation and Corporate Governance

5. Socioeconomic Integration and Modeling

[01:52:55] 5.1 Economic Policy and Public Sentiment Modeling

[01:55:21] 5.2 Free Energy Principle: Libertarian vs Collectivist Perspectives

[01:58:53] 5.3 Regulation of Complex Socio-Technical Systems

[02:03:04] 5.4 Evolution and Current State of Active Inference Research

6. Future Directions and Applications

[02:14:26] 6.1 Active Inference Applications and Future Development

[02:22:58] 6.2 Cultural Learning and Active Inference

[02:29:19] 6.3 Hierarchical Relationship Between FEP, Active Inference, and Bayesian Mechanics

[02:33:22] 6.4 Historical Evolution of Free Energy Principle

[02:38:52] 6.5 Active Inference vs Traditional Machine Learning Approaches

Transcript and shownotes with refs and URLs:

https://www.dropbox.com/scl/fi/qj22a660cob1795ej0gbw/SanjeevShow.pdf?rlkey=w323r3e8zfsnve22caayzb17k&st=el1fdgfr&dl=0