The Gradient: Perspectives on AI

Venkatesh Rao: Protocols, Intelligence, and Scaling

Mar 7, 2024

Explore the nuanced relationship between AI and society with Venkatesh Rao, discussing voice, time in AI systems, abstract planning, and reasoning. Dive into the intersection of data, intelligence, and reasoning in AI, comparing AI learning to human processes. Discover the importance of protocols in AI scaling, privacy challenges, and evolution of mathematical tools in AI. Delve into real-world AI applications, coordination problems in ML, and the need for real-time commitments.

02:18:35

Creator website

Episode guests

Venkatesh Rao

AI Summary

Highlights

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Reevaluation of conceptions of intelligence in AI behavior prompts critical reflections on the dangers of unhinged notions of omnipotent generality.

Importance of embodiment and multi-agent systems in shaping AI interactions underscores the significance of understanding AI's future societal impacts.

Protocols emerge as vital guides in steering AI trajectory, emphasizing the need to consider and prioritize protocol development in AI systems.

Exploration of AI's evolutionary complexities navigates the nuances of personhood, theoretical frameworks, and shifting paradigms crucial for shaping AI trajectories.

Biases in data-driven AI systems intensify as computation simplifies, underscoring the necessity of managing biases inherent in training data for more accurate outcomes.

Deep dives

Reckoning With AI's Behavior

The podcast delves into the complex and intriguing aspects of AI's behavior, prompting a reevaluation of our conceptions of intelligence and its implications. As AI systems become harder to explain, familiar terms lose their clarity, sparking discussions on how we envision the future of AI development and our interactions with these systems.

Evolution of protocols reflects inherent values and purposes

00:42

Longitudinal Interests Shape Distinctive Voice

01:09

Extraordinary artistic abilities in individuals with severe cognitive challenges

01:14

Physics Doesn't Require Formal Training

01:25

Introduction

2min

Evolution of Writing Style and Creative Process

3min

Exploration of Voice and Time in AI Systems Design

16min

Transition from Academic Toy Problems to Real-world Engineering Challenges

32min

Exploring Abstraction Hierarchies through Telescope Analogy

3min

Exploring the Intersection of Data, Intelligence, and Reasoning in AI

23min

The Evolution of Learning: AI vs Human

19min

Exploring Protocols, Intelligence, and Scaling in Various Contexts

3min

Navigating the Intersection of AI, Protocols, and Privacy

28min

10.

Evolution of Mathematical and Programming Tools

6min

11.

Exploring AI in Real-time and Coordination Problems in ML and Protocols

4min

“There is this move from generality in a relative sense of ‘we are not as specialized as insects’ to generality in the sense of omnipotent, omniscient, godlike capabilities. And I think there's something very dangerous that happens there, which is you start thinking of the word ‘general’ in completely unhinged ways.”

In episode 114 of The Gradient Podcast, Daniel Bashir speaks to Venkatesh Rao.

Venkatesh is a writer and consultant. He has been writing the widely read Ribbonfarm blog since 2007, and more recently, the popular Ribbonfarm Studio Substack newsletter. He is the author of Tempo, a book on timing and decision-making, and is currently working on his second book, on the foundations of temporality. He has been an independent consultant since 2011, supporting senior executives in the technology industry. His work in recent years has focused on AI, semiconductor, sustainability, and protocol technology sectors. He holds a PhD in control theory (2003) from the University of Michigan. He is currently based in the Seattle area, and enjoys dabbling in robotics in his spare time. You can learn more about his work at venkateshrao.com

Have suggestions for future podcast guests (or other feedback)? Let us know here or reach us at editor@thegradient.pub

Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSSFollow The Gradient on Twitter

Outline:

* (00:00) Intro

* (01:38) Origins of Ribbonfarm and Venkat’s academic background

* (04:23) Voice and recurring themes in Venkat’s work

* (11:45) Patch models and multi-agent systems: integrating philosophy of language, balancing realism with tractability

* (21:00) More on abstractions vs. tractability in Venkat’s work

* (29:07) Scaling of industrial value systems, characterizing AI as a discipline

* (39:25) Emergent science, intelligence and abstractions, presuppositions in science, generality and universality, cameras and engines

* (55:05) Psychometric terms

* (1:09:07) Inductive biases (yes I mentioned the No Free Lunch Theorem and then just talked about the definition of inductive bias and not the actual theorem 🤡)

* (1:18:13) LLM training and efficiency, comparing LLMs to humans

* (1:23:35) Experiential age, analogies for knowledge transfer

* (1:30:50) More clarification on the analogy

* (1:37:20) Massed Muddler Intelligence and protocols

* (1:38:40) Introducing protocols and the Summer of protocols

* (1:49:15) Evolution of protocols, hardness

* (1:54:20) LLMs, protocols, time, future visions, and progress

* (2:01:33) Protocols, drifting from value systems, friction, compiling explicit knowledge

* (2:14:23) Directions for ML people in protocols research

* (2:18:05) Outro

Links:

* Venkat’s Twitter and homepage