Hungry Hungry Hippos - H3
Deep Papers
Using State-Spaced Models in Language Modeling
The state-space model is actually quite, quite old. They kind of look like either linear RNN or long convolution. And now we're trying to figure out how to put them on par with attention in the realm of language modeling. The S4 model seem to be doing really great at some of these long-range benchmark tasks.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
