Scaling Theory

My guest is Arvind Narayanan, a Professor of Computer Science at Princeton University, and the director of the Center for Information Technology Policy, also at Princeton. Arvind is renowned for his work on the societal impacts of digital technologies, including his textbook on fairness and machine learning, his online course on cryptocurrencies, his research on data de-anonymization, dark patterns, and more. He has already amassed over 30,000 citations on Google Scholar. In just a few days, in late September 2024, Arvind will release a book co-authored with Sayash Kapoor titled “AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference.” Having had the privilege of reading an early version, our conversation delves into some of the book’s key arguments. We also explore what Arvind calls AI scaling myths, the reality of artificial general intelligence, how governments can scale effective AI policies, the importance of transparency, the role that antitrust can, and cannot play, the societal impacts of scaling automation, and more. I hope you enjoy our conversation. Find me on X at @⁠⁠⁠ProfSchrepel⁠⁠⁠⁠⁠⁠. Also, be sure to subscribe. ** References: ➝ AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference (2024) ➝ AI scaling myths (2024) ➝ AI existential risk probabilities are too unreliable to inform policy (2024) ➝ Foundation Model Transparency Reports (2024)

Sara Hooker, VP of Research at Cohere and a recognized AI innovator, shares insights on scaling laws and their limits, emphasizing how smaller models can outperform larger ones. She discusses the balance between open-source and proprietary models, highlighting the need for inclusivity, particularly for multilingual capabilities. Sara also tackles data accessibility challenges and copyright issues affecting AI training, and reflects on how her diverse upbringing informs her approach to innovative research practices. Expect a thought-provoking conversation on AI's future!

Michael Mauboussin, finance expert, talks about increasing returns, market dynamics, Charles Darwin, AI, and the Santa Fe Institute. They discuss leveraging increasing returns in business, impact of intangible assets, navigating complex technologies, and embracing complexity in everyday life.

Economist Glen Weyl discusses his new book on plurality and technology's role in societal change. Topics include open-source AI, blockchain, challenges in open-source compliance, and scaling plurality to address systemic inequalities.

Yann LeCun, a leading computer scientist, discusses AI dynamics and regulation. Topics include open source's role in AI development, scaling AI models, market dynamics, Meta's long-term strategy, the impact of chip manufacturers on AI companies, and the European AI Act. The conversation explores the future of AI, regulatory challenges, and the competition in the AI chip market.

J. Doyne Farmer, expert in Complexity Economics, discusses chaos theory's role in scaling theory, predicting the distant future, conflicts with Moore's Law, agent-based modeling, dominating with theories, and casino strategies. Explore the interplay of chaos theory, scaling laws, innovation impact on economic growth, and advancements in agent-based modeling for understanding human behavior and economic predictions.

Thomas Wolf, Co-founder of Hugging Face, discusses the dominance of open-source AI, challenges in scaling foundational models, big tech companies' strategies, and power laws affecting the ecosystem in a complexity science fashion.

Physicist Geoffrey West discusses scaling laws in living organisms, cities, businesses, and technologies. Topics include power laws in mammal heartbeats, patents and crime in cities, technology scaling, and innovation. The conversation also dives into super-linearity in cities and companies, historical impacts on tech giants, and the need for interdisciplinary collaboration to understand scaling dynamics.

In this first episode, Dr. Thibault Schrepel (@⁠ProfSchrepel⁠) introduces “Scaling Theory”, a podcast dedicated to the power laws behind the growth of companies, technologies, legal and living systems. *** References: ➝ Charles Darwin, On the Origin of Species (1859) ➝ Melanie Mitchell, Complexity: A Guided Tour (2011) ➝ Mitchell Waldrop, Complexity The Emerging Science at the Edge of Order and Chaos (2019) ➝ John H. Miller & Scott Page, Complex Adaptive Systems: An Introduction to Computational Models of Social Life (2007) ➝ W. Brian Arthur, Complexity and the Economy (2014) ➝ Geoffrey West, Scale: The Universal Laws of Life and Death in Organisms, Cities and Companies (2017)