Episode 33: Tri Dao, Stanford: On FlashAttention and sparsity, quantization, and efficient inference
Generally Intelligent
The Importance of Hardware in the Operations Field
Do you think it's well known in the operation field that these algorithms actually like often run slower and need to be much more specialized for hardware? Like are other people also doing that? Or is it this is not really a known thing? I think it is known to a subset of folks who actually put these things in use. Nowadays, especially in the last couple of years where large models like large language models train a massive amount of data.I think the importance of engineering of thinking about hardware is now much more widely used.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
