Episode 33: Tri Dao, Stanford: On FlashAttention and sparsity, quantization, and efficient inference
Generally Intelligent
The Evolution of Attention in Language Models
Last year, I spent a bunch of time trying to make attention as fast as possible. So we wanted to understand from an academic perspective, when or why do we need attention? Can we have other alternatives that maybe scale better in terms of sequence length? Because the longer context length has been a big problem for attention for a long time. And since then, Dan Fue and Michael Paulies have been pushing on that direction. Some things like long convolution, hyena, hierarchy,. They've seen pretty positive results. But it's definitely more of a contrarian take.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
