Distinction Between Programmer-Structured Systems and Learning Systems

17min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

The chapter discusses the contrast between systems where the programmer designs the structure and those where the system learns from data, highlighting the limitations of static intelligence versus learning through data. It explores the potential for algorithmic creativity in unconstrained search spaces like genetic algorithms and delves into how Large Language Models (LLMs) map tokens into a vector space to determine semantic similarity. The chapter also covers Hebbian learning in neural networks like transformers and the use of gradient descent to fit curves and organize tokens on a manifold in LLMs.

Which is more intelligent, ChatGPT or a 3-year old? Of course this depends on what we mean by "intelligence." A modern LLM is certainly able to answer all sorts of questions that require knowledge far past the capacity of a 3-year old, and even to perform synthetic tasks that seem remarkable to many human grown-ups. But is that really intelligence? François Chollet argues that it is not, and that LLMs are not ever going to be truly "intelligent" in the usual sense -- although other approaches to AI might get there.

Support Mindscape on Patreon.

Blog post with transcript: https://www.preposterousuniverse.com/podcast/2024/06/24/280-francois-chollet-on-deep-learning-and-the-meaning-of-intelligence/

François Chollet received his Diplôme d'Ingénieur from École Nationale Supérieure de Techniques Avancées, Paris. He is currently a Senior Staff Engineer at Google. He has been awarded the Global Swiss AI award for breakthroughs in artificial intelligence. He is the author of Deep Learning with Python, and developer of the Keras software library for neural networks. He is the creator of the ARC (Abstraction and Reasoning Corpus) Challenge.

See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.