Deep Papers cover image

Hungry Hungry Hippos - H3

Deep Papers

00:00

Using State-Spaced Models in Language Modeling

The state-space model is actually quite, quite old. They kind of look like either linear RNN or long convolution. And now we're trying to figure out how to put them on par with attention in the realm of language modeling. The S4 model seem to be doing really great at some of these long-range benchmark tasks.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app