2min chapter

The Rhys Show cover image

Neuroscientist Konrad Kording reveals shocking truth about machine learning and the brain

The Rhys Show

CHAPTER

The Parallel Processing of Transformers

The transformers are an example of a deconstraint. The update that you make after using the transformer is in a good way localized. So therefore what I do on a given Transformer I can relatively readily change Now if on the other hand, I have an unended run through things I The update that I need to make is much less localized And I believe that that's one of the things that contribute to the scaling advantages um That's great. Yeah Okay, let me see how to best um Yeah, so Was a while ago that eroded Um, you're good so so You you locally the transformer locally looks at a bunch of tokensUm, it doesn't look at all tokens and

00:00

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode