3min chapter

The Robot Brains Podcast cover image

Geoff Hinton on revolutionizing artificial intelligence... again

The Robot Brains Podcast

CHAPTER

Is There a Way to Distilate Knowledge From One Location to Another?

Convolutional lets do that, and transformers do that too. I don't think the brain can do that, because that would involve weight sharing - it would involve doing exactly the same computation at each locality so you can use the same weights. But actually, there's a way to achieve what weight sharing does next to convolution in a much more plausible way than i think people have suggested before. If you do have contextural predictions trying to agree with locally extracted things, then imagine a whole bunch of columns that are making local predictions and looking at nearby columns to get their contextural prediction. You can think of the text as a teacher for the local thing, but also vice versa

00:00

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode