Prof. Melanie Mitchell, Davis Professor of Complexity at the Santa Fe Institute, dives into the murky waters of AI understanding. She argues that current benchmarks are inadequate, as machines often replicate human tasks without true comprehension. Mitchell highlights the limitations of large language models, noting their lack of common sense despite impressive statistical capabilities. She emphasizes the need for evolving evaluation methods and suggests a deeper, context-specific look at intelligence, advocating for more rigorous testing to reflect genuine understanding.
01:01:47
forum Ask episode
web_stories AI Snips
view_agenda Chapters
menu_book Books
auto_awesome Transcript
info_circle Episode notes
insights INSIGHT
Nature of Intelligence
Intelligence is ill-defined and multidimensional, not a simple yes/no.
AI systems might possess intelligence in specific ways or degrees, rather than absolute intelligence.
question_answer ANECDOTE
Grounded Understanding
Dilip George argues a university professor understands vectors better than AI.
This is because their knowledge is grounded in real-world situations, not just abstract concepts.
insights INSIGHT
Understanding in LLMs
Key question: Is AI understanding a category error, mistaking token associations for real-world connections?
Or do large language models create concept-based mental models like humans, and does scaling improve them?
Get the Snipd Podcast app to discover more snips from this episode
In 'Complexity: A Guided Tour', Melanie Mitchell provides a comprehensive overview of complex systems science, covering topics such as chaos theory, genetic algorithms, and network theory. The book explores how complex, organized behavior arises from simple interactions across various biological, technological, and social phenomena. It offers insights into the current research and future prospects of complexity science.
Gödel, Escher, Bach
An Eternal Golden Braid
Douglas Hofstadter
This book by Douglas Hofstadter is a comprehensive and interdisciplinary work that explores the interrelated ideas of Kurt Gödel, M.C. Escher, and Johann Sebastian Bach. It delves into concepts such as self-reference, recursion, and the limits of formal systems, particularly through Gödel's Incompleteness Theorem. The book uses dialogues between fictional characters, including Achilles and the Tortoise, to intuitively present complex ideas before they are formally explained. It covers a wide range of topics including cognitive science, artificial intelligence, number theory, and the philosophy of mind, aiming to understand how consciousness and intelligence emerge from formal systems[2][4][5].
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CB
Prof. Melanie Mitchell argues that the concept of "understanding" in AI is ill-defined and multidimensional - we can't simply say an AI system does or doesn't understand. She advocates for rigorously testing AI systems' capabilities using proper experimental methods from cognitive science. Popular benchmarks for intelligence often rely on the assumption that if a human can perform a task, an AI that performs the task must have human-like general intelligence. But benchmarks should evolve as capabilities improve.
Large language models show surprising skill on many human tasks but lack common sense and fail at simple things young children can do. Their knowledge comes from statistical relationships in text, not grounded concepts about the world. We don't know if their internal representations actually align with human-like concepts. More granular testing focused on generalization is needed.
There are open questions around whether large models' abilities constitute a fundamentally different non-human form of intelligence based on vast statistical correlations across text. Mitchell argues intelligence is situated, domain-specific and grounded in physical experience and evolution. The brain computes but in a specialized way honed by evolution for controlling the body. Extracting "pure" intelligence may not work.
Other key points:
- Need more focus on proper experimental method in AI research. Developmental psychology offers examples for rigorous testing of cognition.
- Reporting instance-level failures rather than just aggregate accuracy can provide insights.
- Scaling laws and complex systems science are an interesting area of complexity theory, with applications to understanding cities.
- Concepts like "understanding" and "intelligence" in AI force refinement of fuzzy definitions.
- Human intelligence may be more collective and social than we realize. AI forces us to rethink concepts we apply anthropomorphically.
The overall emphasis is on rigorously building the science of machine cognition through proper experimentation and benchmarking as we assess emerging capabilities.
TOC:
[00:00:00] Introduction and Munk AI Risk Debate Highlights
[05:00:00] Douglas Hofstadter on AI Risk
[00:06:56] The Complexity of Defining Intelligence
[00:11:20] Examining Understanding in AI Models
[00:16:48] Melanie's Insights on AI Understanding Debate
[00:22:23] Unveiling the Concept Arc
[00:27:57] AI Goals: A Human vs Machine Perspective
[00:31:10] Addressing the Extrapolation Challenge in AI
[00:36:05] Brain Computation: The Human-AI Parallel
[00:38:20] The Arc Challenge: Implications and Insights
[00:43:20] The Need for Detailed AI Performance Reporting
[00:44:31] Exploring Scaling in Complexity Theory
Eratta:
Note Tim said around 39 mins that a recent Stanford/DM paper modelling ARC “on GPT-4 got around 60%”. This is not correct and he misremembered. It was actually davinci3, and around 10%, which is still extremely good for a blank slate approach with an LLM and no ARC specific knowledge. Folks on our forum couldn’t reproduce the result. See paper linked below.
Books (MUST READ):
Artificial Intelligence: A Guide for Thinking Humans (Melanie Mitchell)
https://www.amazon.co.uk/Artificial-Intelligence-Guide-Thinking-Humans/dp/B07YBHNM1C/?&_encoding=UTF8&tag=mlst00-21&linkCode=ur2&linkId=44ccac78973f47e59d745e94967c0f30&camp=1634&creative=6738
Complexity: A Guided Tour (Melanie Mitchell)
https://www.amazon.co.uk/Audible-Complexity-A-Guided-Tour?&_encoding=UTF8&tag=mlst00-21&linkCode=ur2&linkId=3f8bd505d86865c50c02dd7f10b27c05&camp=1634&creative=6738
Machine Learning Street Talk (MLST) 
