Prof. Emily Bender and Pablo Arredondo discuss the intelligence of generative AI systems like ChatGPT and their role in legal reasoning. The limitations of chatbots in language understanding and the concept of 'stochastic parrots' are explored. The challenges of AI systems in explaining reasoning and the debate on AGI funding and anthropomorphization are also discussed.
Generative AI systems like chat GPT excel at pattern matching but lack true comprehension and understanding.
Claims of emergent capabilities and impressive reasoning abilities in AI systems may be misleading and not indicative of true intelligence.
Deep dives
The Limitations of AI and the Clever Hans Effect
Artificial intelligence, particularly generative AI systems like chat GPT, may appear to exhibit human-level cognitive abilities, such as reasoning and problem-solving. However, experts argue that these systems are actually engaged in advanced pattern matching rather than true understanding or intelligence. They can generate plausible text and even pass exams like the bar exam with impressive scores, but their lack of true comprehension is revealed when asked to explain their reasoning. These AI systems, like chat GPT, are essentially large language models that excel at stringing words together in a coherent manner, but they lack the deeper understanding and meaning behind the words. This reliance on superficial patterns and cues from training data, known as the clever Hans effect, can lead to correct answers for the wrong reasons.
The Hype and Misunderstanding Surrounding AI
The claims and expectations surrounding artificial general intelligence (AGI), or human-level cognition in machines, may be overstated. AI systems like chat GPT are often marketed with claims of emergent capabilities and impressive reasoning abilities. However, researchers suggest that these claims can be misleading, as the systems are primarily engaged in automated pattern matching and lack true understanding or cognitive processes. There is debate surrounding the definition and achievement of true intelligence in machines, and whether the path to AGI lies in simply scaling up existing models or involves additional elements that are not yet understood. The enthusiasm and emotional investment in achieving AGI may be clouding the scientific assessment of AI systems and their actual capabilities.
Questioning the Future of AI and Human-Like Reasoning
While AI systems like chat GPT can perform specific reasoning tasks and simulate human-like conversation, they fall short in more complex reasoning and inference requiring genuine human-like thought and understanding. The current models heavily rely on vast amounts of training data to generate coherent responses, but they lack the true internal thought processes and comprehension that humans possess. Researchers who have tested the systems on various tasks and scenarios highlight the limitations and potential pitfalls of relying on AI for human-level intelligence. The future trajectory of AI and the achievement of artificial general intelligence remain uncertain, and there are doubts about whether current models can truly replicate human reasoning and cognition.
Are generative AI systems such as OpenAI’s ChatGPT really intelligent? Large language models such as GPT 4 appear to use human-level cognitive abilities when they engage in legal reasoning, write essays or solve complex problems. Hosts John Thornhill and Madhumita Murgia speak to Emily Bender, professor of computational linguistics at the University of Washington, to find out what’s really happening under the hood, and also hear from Pablo Arredondo of CaseText, which develops AI tools for lawyers; influential computer scientist Melanie Mitchell, professor at the Santa Fe Institute, and Konstantine Arkoudas, an AI expert who’s worked on Amazon’s Alexa.
Tech Tonic is presented by Madhumita Murgia and John Thornhill. Senior producer is Edwin Lane and the producer is Josh Gabert-Doyon. Executive producer is Manuela Saragosa. Sound design by Breen Turner and Samantha Giovinco. Original music by Metaphor Music. The FT’s head of audio is Cheryl Brumley.
