AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
The focus of the podcast episode delves into distinguishing between theories of intelligence that emphasize thinking versus knowing. It highlights the importance of understanding how much knowledge individuals possess, especially in the context of artificial intelligence systems.
Neural networks are discussed as large systems capable of learning functions through tunable weights. The power of neural networks lies in their ability to represent data and knowledge through internal structures, allowing them to recognize similarities, differences, relations, and abstractions between objects and concepts.
The podcast emphasizes the significance of structured knowledge over sheer volume, as illustrated by examples of individuals with exceptional memory abilities. It is highlighted that the organizational structure of knowledge is crucial for its utility, rather than the sheer amount of information one can retain.
Creative outputs from AI systems like AlphaGo are showcased, prompting discussions on what defines creativity and the relationship between generative language models and human prompting. The podcast explores whether AI systems need agency to exhibit creativity and how in-context learning contributes to emergent creative problem-solving.
Grounding AI performance and training in human learning, beliefs, and preferences improves usability. Constraint with human data can align AI systems better with human capabilities and preferences. AI systems need to understand and cater to human needs and beliefs for practical application.
Modeling the real world with AI frameworks faces challenges like intrinsic rewards. In real-world scenarios, rewards are tied to intrinsic experiences, not externally administered. AI systems need to adapt to local circumstances and constantly form models of the world based on observations, unlike standard deep learning training. The mismatch between deep learning principles and real-world dynamics hinders generalizability of AI systems.
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5
Christopher Summerfield, Department of Experimental Psychology, University of Oxford is a Professor of Cognitive Neuroscience at the University of Oxford and a Research Scientist at Deepmind UK. His work focusses on the neural and computational mechanisms by which humans make decisions.
Chris has just released an incredible new book on AI called "Natural General Intelligence". It's my favourite book on AI I have read so so far.
The book explores the algorithms and architectures that are driving progress in AI research, and discusses intelligence in the language of psychology and biology, using examples and analogies to be comprehensible to a wide audience. It also tackles longstanding theoretical questions about the nature of thought and knowledge.
With Chris' permission, I read out a summarised version of Chapter 2 from his book on which was on Intelligence during the 30 minute MLST introduction.
Buy his book here:
https://global.oup.com/academic/product/natural-general-intelligence-9780192843883?cc=gb&lang=en&
YT version: https://youtu.be/31VRbxAl3t0
Interviewer: Dr. Tim Scarfe
TOC:
[00:00:00] Walk and talk with Chris on Knowledge and Abstractions
[00:04:08] Intro to Chris and his book
[00:05:55] (Intro) Tim reads Chapter 2: Intelligence
[00:09:28] Intro continued: Goodhart's law
[00:15:37] Intro continued: The "swiss cheese" situation
[00:20:23] Intro continued: On Human Knowledge
[00:23:37] Intro continued: Neats and Scruffies
[00:30:22] Interview kick off
[00:31:59] What does it mean to understand?
[00:36:18] Aligning our language models
[00:40:17] Creativity
[00:41:40] "Meta" AI and basins of attraction
[00:51:23] What can Neuroscience impart to AI
[00:54:43] Sutton, neats and scruffies and human alignment
[01:02:05] Reward is enough
[01:19:46] Jon Von Neumann and Intelligence
[01:23:56] Compositionality
References:
The Language Game (Morten H. Christiansen, Nick Chater
https://www.penguin.co.uk/books/441689/the-language-game-by-morten-h-christiansen-and--nick-chater/9781787633483
Theory of general factor (Spearman)
https://www.proquest.com/openview/7c2c7dd23910c89e1fc401e8bb37c3d0/1?pq-origsite=gscholar&cbl=1818401
Intelligence Reframed (Howard Gardner)
https://books.google.co.uk/books?hl=en&lr=&id=Qkw4DgAAQBAJ&oi=fnd&pg=PT6&dq=howard+gardner+multiple+intelligences&ots=ERUU0u5Usq&sig=XqiDgNUIkb3K9XBq0vNbFmXWKFs#v=onepage&q=howard%20gardner%20multiple%20intelligences&f=false
The master algorithm (Pedro Domingos)
https://www.amazon.co.uk/Master-Algorithm-Ultimate-Learning-Machine/dp/0241004543
A Thousand Brains: A New Theory of Intelligence (Jeff Hawkins)
https://www.amazon.co.uk/Thousand-Brains-New-Theory-Intelligence/dp/1541675819
The bitter lesson (Rich Sutton)
http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Listen to all your favourite podcasts with AI-powered features
Listen to the best highlights from the podcasts you love and dive into the full episode
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
Listen to all your favourite podcasts with AI-powered features
Listen to the best highlights from the podcasts you love and dive into the full episode