Generally Intelligent cover image

Episode 33: Tri Dao, Stanford: On FlashAttention and sparsity, quantization, and efficient inference

Generally Intelligent

00:00

The Future of Recurrent Networks

I think transformers are here to stay at least for a while. Lots of infrastructure even on the level of software framework and hardware are very much tailored to transformers. So I don't think anything is going to replace, completely replace them very soon. But we'll see new approaches coming out simply because there are now new applications that require either long context or reasoning ability.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app