Support us! https://www.patreon.com/mlst Dr. Walid Saba recently reviewed the book Machines Will Never Rule The World, which argues that strong AI is impossible. He acknowledges the complexity of modeling mental processes and language, as well as interactive dialogues, and questions the authors' use of "never." Despite his skepticism, he is impressed with recent developments in large language models, though he questions the extent of their success. We then discussed the successes of cognitive science. Walid believes that something has been achieved which many cognitive scientists would never accept, namely the ability to learn from data empirically. Keith agrees that this is a huge step, but notes that there is still much work to be done to get to the "other 5%" of accuracy. They both agree that the current models are too brittle and require much more data and parameters to get to the desired level of accuracy. Walid then expresses admiration for deep learning systems' ability to learn non-trivial aspects of language from ingesting text only. He argues that this is an "existential proof" of language competency and that it would be impossible for a group of luminaries such as Montague, Marvin Minsky, John McCarthy, and a thousand other bright engineers to replicate the same level of competency as we have now with LLMs. He then discusses the problem of semantics and pragmatics, as well as symbol grounding, and expresses skepticism about grounded meaning and embodiment. He believes that artificial intelligence should be used to solve real-world problems which require human intelligence but not believe that robots should be built to understand love or other subjective feelings. We discussed the unique properties of natural human language. Walid believes that the core unique property is the ability to do abductive reasoning, which is the process of reasoning to the best explanation or understanding. Keith adds that there are two types of abduction - one for generating hypotheses and one for justifying them. In both cases, abductive reasoning involves choosing from a set of plausible possibilities. Finally, we discussed the book "Machines Will Never Rule The World" and its argument that the current mathematics and technology is not enough to model complex systems. Walid agrees with the book's argument but is still optimistic that a new mathematics can be discovered. Keith suggests the possibility of an AGI discovering the mathematics to create itself. They also discussed how the book could serve as a reminder to temper the hype surrounding AI and to focus on exploration, creativity, and daring ideas. Walid ended by stressing the importance of science, noting that engineers should play within the Venn diagrams drawn by scientists, rather than trying to hack their way through it. Transcript: https://share.descript.com/view/BFQb5iaegJC Discord: https://discord.gg/aNPkGUQtc5 YT: https://youtu.be/IMnWAuoucjo TOC: [00:00:00] Intro [00:06:52] Walid's change of heart on DL/LLMs and on the skeptics like Gary Marcus [00:22:52] Symbol Grounding [00:32:26] On Montague [00:40:41] On Abduction [00:50:54] Language of thought [00:56:08] Why machines will never rule the world book review [01:20:06] Engineers should play in the scientists Venn Diagram! Panel; Dr. Tim Scarfe Dr. Keith Duggar Mark Mcguill

Yann LeCun is a French computer scientist known for his pioneering work on convolutional neural networks, optical character recognition and computer vision. He is a Silver Professor at New York University and Vice President, Chief AI Scientist at Meta. Along with Yoshua Bengio and Geoffrey Hinton, he was awarded the 2018 Turing Award for their work on deep learning, earning them the nickname of the "Godfathers of Deep Learning". Dr. Randall Balestriero has been researching learnable signal processing since 2013, with a focus on learnable parametrized wavelets and deep wavelet transforms. His research has been used by NASA, leading to applications such as Marsquake detection. During his PhD at Rice University, Randall explored deep networks from a theoretical perspective and improved state-of-the-art methods such as batch-normalization and generative networks. Later, when joining Meta AI Research (FAIR) as a postdoc with Prof. Yann LeCun, Randall further broadened his research interests to include self-supervised learning and the biases emerging from data-augmentation and regularization, resulting in numerous publications. Episode recorded live at NeurIPS.  YT: https://youtu.be/9dLd6n9yT8U (references are there) Support us! https://www.patreon.com/mlst  Host: Dr. Tim Scarfe TOC: [00:00:00] LeCun interview [00:18:25] Randall Balestriero interview (mostly on spectral SSL paper, first ref)

Dr. Petar Veličković  is a Staff Research Scientist at DeepMind, he has firmly established himself as one of the most significant up and coming researchers in the deep learning space. He invented Graph Attention Networks in 2017 and has been a leading light in the field ever since pioneering research in Graph Neural Networks, Geometric Deep Learning and also Neural Algorithmic reasoning. If you haven’t already, you should check out our video on the Geometric Deep learning blueprint, featuring Petar. I caught up with him last week at NeurIPS. In this show, from NeurIPS 2022 we discussed his recent work on category theory and graph neural networks. https://petar-v.com/ https://twitter.com/PetarV_93/ TOC: Categories  (Cats for AI) [00:00:00] Reasoning [00:14:44] Extrapolation [00:19:09] Ishan Misra Skit [00:27:50] Graphs (Expander Graph Propagation) [00:29:18] YT: https://youtu.be/1lkdWduuN14 MLST Discord: https://discord.gg/V25vQeFwhS Support us! https://www.patreon.com/mlst References on YT description, lots of them!  Host: Dr. Tim Scarfe

In this NeurIPSs interview, we speak with Laura Ruis about her research on the ability of language models to interpret language in context. She has designed a simple task to evaluate the performance of widely used state-of-the-art language models and has found that they struggle to make pragmatic inferences (implicatures). Tune in to learn more about her findings and what they mean for the future of conversational AI. Laura Ruis https://www.lauraruis.com/ https://twitter.com/LauraRuis BLOOM https://bigscience.huggingface.co/blog/bloom Large language models are not zero-shot communicators [Laura Ruis, Akbir Khan, Stella Biderman, Sara Hooker, Tim Rocktäschel, Edward Grefenstette] https://arxiv.org/abs/2210.14986 [Zhang et al] OPT: Open Pre-trained Transformer Language Models https://arxiv.org/pdf/2205.01068.pdf [Lampinen] Can language models handle recursively nested grammatical structures? A case study on comparing models and humans https://arxiv.org/pdf/2210.15303.pdf [Gary Marcus] Horse rides astronaut https://garymarcus.substack.com/p/horse-rides-astronaut [Gary Marcus] GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about https://www.technologyreview.com/2020/08/22/1007539/gpt3-openai-language-generator-artificial-intelligence-ai-opinion/ [Bender et al] On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? https://dl.acm.org/doi/10.1145/3442188.3445922  [janus] Simulators (Less Wrong) https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators

First in our unplugged series live from #NeurIPS2022 We discuss natural language understanding, symbol meaning and grounding and Chomsky with Dr. Andrew Lampinen from DeepMind.  We recorded a LOT of material from NeurIPS, keep an eye out for the uploads.  YT version: https://youtu.be/46A-BcBbMnA References [Paul Cisek] Beyond the computer metaphor: Behaviour as interaction https://philpapers.org/rec/CISBTC Linguistic Competence (Chomsky reference) https://en.wikipedia.org/wiki/Linguistic_competence [Andrew Lampinen] Can language models handle recursively nested grammatical structures? A case study on comparing models and humans https://arxiv.org/abs/2210.15303 [Fodor et al] Connectionism and Cognitive Architecture: A Critical Analysis https://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/proseminars/Proseminar13/ConnectionistArchitecture.pdf [Melanie Mitchell et al] The Debate Over Understanding in AI's Large Language Models https://arxiv.org/abs/2210.13966 [Gary Marcus] GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about https://www.technologyreview.com/2020/08/22/1007539/gpt3-openai-language-generator-artificial-intelligence-ai-opinion/ [Bender et al] On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? https://dl.acm.org/doi/10.1145/3442188.3445922 [Adam Santoro, Andrew Lampinen et al] Symbolic Behaviour in Artificial Intelligence https://arxiv.org/abs/2102.03406 [Ishita Dasgupta, Lampinen et al] Language models show human-like content effects on reasoning https://arxiv.org/abs/2207.07051 REACT - Synergizing Reasoning and Acting in Language Models https://arxiv.org/pdf/2210.03629.pdf https://ai.googleblog.com/2022/11/react-synergizing-reasoning-and-acting.html [Fabian Paischer] HELM - History Compression via Language Models in Reinforcement Learning https://ml-jku.github.io/blog/2022/helm/ https://arxiv.org/abs/2205.12258 [Laura Ruis] Large language models are not zero-shot communicators https://arxiv.org/pdf/2210.14986.pdf [Kumar] Using natural language and program abstractions to instill human inductive biases in machines https://arxiv.org/pdf/2205.11558.pdf Juho Kim https://juhokim.com/

AI Helps Ukraine - Charity Conference A charity conference on AI to raise funds for medical and humanitarian aid for Ukraine https://aihelpsukraine.cc/ YT version: https://youtu.be/LgwjcqhkOA4 Support us! https://www.patreon.com/mlst  Dr. Joscha Bach (born 1973 in Weimar, Germany) is a German artificial intelligence researcher and cognitive scientist focusing on cognitive architectures, mental representation, emotion, social modelling, and multi-agent systems.  http://bach.ai/ https://twitter.com/plinz TOC: [00:00:00] Ukraine Charity Conference and NeurIPS 2022 [00:03:40] Theory of computation, Godel, Penrose [00:11:44] Modelling physical reality [00:15:19] Is our universe infinite? [00:24:30] Large language models, and on DL / is Gary Marcus hitting a wall? [00:45:17] Generative models / Codex / Language of thought [00:58:46] Consciousness (with Friston references) References: Am I Self-Conscious? (Or Does Self-Organization Entail Self-Consciousness?) [Friston] https://www.frontiersin.org/articles/10.3389/fpsyg.2018.00579/full Impact of Pretraining Term Frequencies on Few-Shot Reasoning [Yasaman Razeghi] https://arxiv.org/abs/2202.07206 Deep Learning Is Hitting a Wall [Gary Marcus] https://nautil.us/deep-learning-is-hitting-a-wall-238440/ Turing machines https://en.wikipedia.org/wiki/Turing_machine Lambda Calculus https://en.wikipedia.org/wiki/Lambda_calculus Godel's incompletness theorem https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems Oracle machine https://en.wikipedia.org/wiki/Oracle_machine

Support us (and please rate on podcast app) https://www.patreon.com/mlst  In this show tonight with Prof. Julian Togelius (NYU) and Prof. Ken Stanley we discuss open-endedness, AGI, game AI and reinforcement learning.   [Prof Julian Togelius] https://engineering.nyu.edu/faculty/julian-togelius https://twitter.com/togelius [Prof Ken Stanley] https://www.cs.ucf.edu/~kstanley/ https://twitter.com/kenneth0stanley TOC: [00:00:00] Introduction [00:01:07] AI and computer games [00:12:23] Intelligence [00:21:27] Intelligence Explosion [00:25:37] What should we be aspiring towards? [00:29:14] Should AI contribute to culture? [00:32:12] On creativity and open-endedness [00:36:11] RL overfitting [00:44:02] Diversity preservation [00:51:18] Empiricism vs rationalism , in gradient descent the data pushes you around [00:55:49] Creativity and interestingness (does complexity / information increase) [01:03:20] What does a population give us? [01:05:58] Emergence / generalisation snobbery References; [Hutter/Legg] Universal Intelligence: A Definition of Machine Intelligence https://arxiv.org/abs/0712.3329 https://en.wikipedia.org/wiki/Artificial_general_intelligence https://en.wikipedia.org/wiki/I._J._Good https://en.wikipedia.org/wiki/G%C3%B6del_machine [Chollet] Impossibility of intelligence explosion https://medium.com/@francois.chollet/the-impossibility-of-intelligence-explosion-5be4a9eda6ec [Alex Irpan] - RL is hard https://www.alexirpan.com/2018/02/14/rl-hard.html https://nethackchallenge.com/ Map elites https://arxiv.org/abs/1504.04909 Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space https://arxiv.org/abs/1912.02400 [Stanley] - Why greatness cannot be planned https://www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/3319155237 [Lehman/Stanley] Abandoning Objectives: Evolution through the Search for Novelty Alone https://www.cs.swarthmore.edu/~meeden/DevelopmentalRobotics/lehman_ecj11.pdf

We had a conversation with Aidan Gomez, the CEO of language-based AI platform Cohere. Cohere is a startup which uses artificial intelligence to help users build the next generation of language-based applications. It's headquartered in Toronto. The company has raised $175 million in funding so far. Language may well become a key new substrate for software building, both in its representation and how we build the software. It may democratise software building so that more people can build software, and we can build new types of software. Aidan and I discuss this in detail in this episode of MLST. Check out Cohere -- https://dashboard.cohere.ai/welcome/register?utm_source=influencer&utm_medium=social&utm_campaign=mlst Support us! https://www.patreon.com/mlst  YT version: https://youtu.be/ooBt_di8DLs TOC: [00:00:00] Aidan Gomez intro [00:02:12] What's it like being a CEO? [00:02:52] Transformers [00:09:33] Deepmind Chomsky Hierarchy [00:14:58] Cohere roadmap [00:18:18] Friction using LLMs for startups [00:25:31] How different from OpenAI / GPT-3 [00:29:31] Engineering questions on Cohere [00:35:13] Francois Chollet says that LLMs are like databases [00:38:34] Next frontier of language models [00:42:04] Different modes of understanding in LLMs [00:47:04] LLMs are the new extended mind [00:50:03] Is language the next interface, and why might that be bad? References: [Balestriero] Spine theory of NNs https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf [Delétang et al] Neural Networks and the Chomsky Hierarchy https://arxiv.org/abs/2207.02098 [Fodor, Pylyshyn] Connectionism and Cognitive Architecture: A Critical Analysis https://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/docs/jaf.pdf [Chalmers, Clark] The extended mind https://icds.uoregon.edu/wp-content/uploads/2014/06/Clark-and-Chalmers-The-Extended-Mind.pdf [Melanie Mitchell et al] The Debate Over Understanding in AI's Large Language Models https://arxiv.org/abs/2210.13966 [Jay Alammar] Illustrated stable diffusion https://jalammar.github.io/illustrated-stable-diffusion/ Illustrated transformer https://jalammar.github.io/illustrated-transformer/ https://www.youtube.com/channel/UCmOwsoHty5PrmE-3QhUBfPQ [Sandra Kublik] (works at Cohere!) https://www.youtube.com/channel/UCjG6QzmabZrBEeGh3vi-wDQ

This video is demonetised on music copyright so we would appreciate support on our Patreon! https://www.patreon.com/mlst  We would also appreciate it if you rated us on your podcast platform.  YT: https://youtu.be/_KVAzAzO5HU Panel: Dr. Tim Scarfe, Dr. Keith Duggar Guests: Prof. J. Mark Bishop, Francois Chollet, Prof. David Chalmers, Dr. Joscha Bach, Prof. Karl Friston, Alexander Mattick, Sam Roffey The Chinese Room Argument was first proposed by philosopher John Searle in 1980. It is an argument against the possibility of artificial intelligence (AI) – that is, the idea that a machine could ever be truly intelligent, as opposed to just imitating intelligence. The argument goes like this: Imagine a room in which a person sits at a desk, with a book of rules in front of them. This person does not understand Chinese. Someone outside the room passes a piece of paper through a slot in the door. On this paper is a Chinese character. The person in the room consults the book of rules and, following these rules, writes down another Chinese character and passes it back out through the slot. To someone outside the room, it appears that the person in the room is engaging in a conversation in Chinese. In reality, they have no idea what they are doing – they are just following the rules in the book. The Chinese Room Argument is an argument against the idea that a machine could ever be truly intelligent. It is based on the idea that intelligence requires understanding, and that following rules is not the same as understanding. in this detailed investigation into the Chinese Room, Consciousness and Syntax vs Semantics, we interview luminaries J.Mark Bishop and Francois Chollet and use unreleased footage from our interviews with David Chalmers, Joscha Bach and Karl Friston. We also cover material from Walid Saba and interview Alex Mattick from Yannic's Discord.  This is probably my favourite ever episode of MLST. I hope you enjoy it!  With Keith Duggar.  Note that we are using clips from our unreleased interviews from David Chalmers and Joscha Bach -- we will release those shows properly in the coming weeks. We apologise for delay releasing our backlog, we have been busy building a startup company in the background. TOC:  [00:00:00] Kick off [00:00:46] Searle [00:05:09] Bishop introduces CRA [00:00:00] Stevan Hardad take on CRA  [00:14:03] Francois Chollet dissects CRA [00:34:16] Chalmers on consciousness [00:36:27] Joscha Bach on consciousness [00:42:01] Bishop introduction [00:51:51] Karl Friston on consciousness [00:55:19] Bishop on consciousness and comments on Chalmers  [01:21:37] Private language games (including clip with Sam Roffey) [01:27:27] Dr. Walid Saba on the chinese room (gofai/systematicity take) [00:34:36] Bishop: on agency / teleology [01:36:38] Bishop: back to CRA [01:40:53] Noam Chomsky on mysteries  [01:45:56] Eric Curiel on math does not represent [01:48:14] Alexander Mattick on syntax vs semantics Thanks to: Mark MC on Discord for stimulating conversation, Alexander Mattick, Dr. Keith Duggar, Sam Roffey. Sam's YouTube channel is https://www.youtube.com/channel/UCjRNMsglFYFwNsnOWIOgt1Q

Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB In this special edition episode, we have a conversation with Prof. Noam Chomsky, the father of modern linguistics and the most important intellectual of the 20th century.  With a career spanning the better part of a century, we took the chance to ask Prof. Chomsky his thoughts not only on the progress of linguistics and cognitive science but also the deepest enduring mysteries of science and philosophy as a whole - exploring what may lie beyond our limits of understanding. We also discuss the rise of connectionism and large language models, our quest to discover an intelligible world, and the boundaries between silicon and biology. We explore some of the profound misunderstandings of linguistics in general and Chomsky’s own work specifically which have persisted, at the highest levels of academia for over sixty years.   We have produced a significant introduction section where we discuss in detail Yann LeCun’s recent position paper on AGI, a recent paper on emergence in LLMs, empiricism related to cognitive science, cognitive templates, “the ghost in the machine” and language.  Panel:  Dr. Tim Scarfe Dr. Keith Duggar Dr. Walid Saba  YT version: https://youtu.be/-9I4SgkHpcA 00:00:00 Kick off 00:02:24 C1: LeCun's recent position paper on AI, JEPA, Schmidhuber, EBMs 00:48:38 C2: Emergent abilities in LLMs paper 00:51:32 C3: Empiricism 01:25:33 C4: Cognitive Templates 01:35:47 C5: The Ghost in the Machine 01:59:21 C6: Connectionism and Cognitive Architecture: A Critical Analysis by Fodor and Pylyshyn 02:19:25 C7: We deep-faked Chomsky 02:29:11 C8: Language 02:34:41 C9: Chomsky interview kick-off! 02:35:39 Large Language Models such as GPT-3 02:39:14 Connectionism and radical empiricism 02:44:44 Hybrid systems such as neurosymbolic 02:48:47 Computationalism silicon vs biological 02:53:28 Limits of human understanding 03:00:46 Semantics state-of-the-art 03:06:43 Universal grammar, I-Language, and language of thought 03:16:27 Profound and enduring misunderstandings 03:25:41 Greatest remaining mysteries science and philosophy 03:33:10 Debrief and 'Chuckles' from Chomsky