BI 183 Dan Goodman: Neural Reckoning

4 snips

Feb 6, 2024

Dan Goodman, co-founder of Neuromatch and creator of the Brian spiking neural network simulator, discusses the importance of spikes in intelligent systems and the curious choice of modern neural networks to disregard spiking. They delve into the intricacies of spiking neural networks, the transition from math to neuroscience, the creation of complex tasks for neural networks, and the challenges in training them. They also explore the impact of advanced technology on human intelligence.

Ask episode

Chapters

Transcript

Episode notes

Introduction

00:00 • 4min

Exploring the Intricacies of Spiking Neural Networks

03:40 • 2min

Transition to Neuroscience and Comparison between Physicists and Mathematicians

05:12 • 20min

Creating More Complex Tasks for Neural Networks

25:21 • 15min

Heterogeneity and Time Constants in Spiking Neural Networks

39:57 • 16min

Understanding Information Loss and Gain in Autoencoders

55:29 • 2min

Low Dimensional Manifold and Sparsity in Networks

57:14 • 11min

Challenges in Training Spiking Neural Networks

01:08:12 • 12min

The Impact of Advanced Technology on Human Intelligence

01:20:02 • 7min

Concluding the conversation and podcast promotions

01:27:05 • 2min

Support the show to get full episodes, full archive, and join the Discord community.

You may know my guest as the co-founder of Neuromatch, the excellent online computational neuroscience academy, or as the creator of the Brian spiking neural network simulator, which is freely available. I know him as a spiking neural network practitioner extraordinaire. Dan Goodman runs the Neural Reckoning Group at Imperial College London, where they use spiking neural networks to figure out how biological and artificial brains reckon, or compute.

All of the current AI we use to do all the impressive things we do, essentially all of it, is built on artificial neural networks. Notice the word "neural" there. That word is meant to communicate that these artificial networks do stuff the way our brains do stuff. And indeed, if you take a few steps back, spin around 10 times, take a few shots of whiskey, and squint hard enough, there is a passing resemblance. One thing you'll probably still notice, in your drunken stupor, is that, among the thousand ways ANNs differ from brains, is that they don't use action potentials, or spikes. From the perspective of neuroscience, that can seem mighty curious. Because, for decades now, neuroscience has focused on spikes as the things that make our cognition tick.

We count them and compare them in different conditions, and generally put a lot of stock in their usefulness in brains.

So what does it mean that modern neural networks disregard spiking altogether?

Maybe spiking really isn't important to process and transmit information as well as our brains do. Or maybe spiking is one among many ways for intelligent systems to function well. Dan shares some of what he's learned and how he thinks about spiking and SNNs and a host of other topics.

0:00 - Intro 3:47 - Why spiking neural networks, and a mathematical background 13:16 - Efficiency 17:36 - Machine learning for neuroscience 19:38 - Why not jump ship from SNNs? 23:35 - Hard and easy tasks 29:20 - How brains and nets learn 32:50 - Exploratory vs. theory-driven science 37:32 - Static vs. dynamic 39:06 - Heterogeneity 46:01 - Unifying principles vs. a hodgepodge 50:37 - Sparsity 58:05 - Specialization and modularity 1:00:51 - Naturalistic experiments 1:03:41 - Projects for SNN research 1:05:09 - The right level of abstraction 1:07:58 - Obstacles to progress 1:12:30 - Levels of explanation 1:14:51 - What has AI taught neuroscience? 1:22:06 - How has neuroscience helped AI?