AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
1min snip
The State Space Model Revolution, with Albert Gu
"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis
Understanding Mapping and State in Computational Models
In various computational models like attention and Mamba, the fundamental concept is a mapping from inputs to outputs. While attention functions as a sequence-to-sequence mapping without a state in the forward pass, a state becomes crucial during inference to aid in decoding. Models like Mamba still materialize the state but discard it immediately, rematerializing it during the backward pass. Linear attention and SSMs are related in that they both involve a concept of a state, which can be materialized in different ways, such as using a convolution instead of relying on a recurrent state in the training pass.
00:00
Transcript
Episode notes
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode