Chasing Real AGI: Inside ARC Prize 2025 with Chollet & Knoop

16 snips

Apr 3, 2025

Guest

Mike Knoop

Guest

François Chollet

Join François Chollet, a leading AI researcher and philosopher behind Keras, and Mike Knoop, co-founder of Zapier and the ARC Prize, as they explore the future of artificial general intelligence (AGI). They discuss the launch of ARC AGI 2 and how current language models fall short in true intelligence benchmarks. Tune in to hear about advancements like the O3 model, the significance of test-time adaptation, and the philosophical underpinnings of intelligence. Their new research lab, Ndea, aims to revolutionize AI and foster rapid scientific progress.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

ARC's Focus: Fluid Intelligence

ARC measures AI fluid intelligence, unlike other benchmarks that focus on specific skills.
It uses novel tasks to evaluate how well AI adapts, a current weakness.

INSIGHT

Shift to Test-Time Adaptation

The AI research world shifted from static LLMs to test-time adaptation.
Models like O3, with longer latency and higher cost, show stronger generalization.

ANECDOTE

LLM Scaling Limitations

From GPT-2 to GPT-4, a 50,000x scale-up resulted in almost no improvement on ARC.
ARC resists the pre-training scaling paradigm, highlighting the need for different approaches.

Get the Snipd Podcast app to discover more snips from this episode

Get the app