AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
3min snip
Episode 33: Tri Dao, Stanford: On FlashAttention and sparsity, quantization, and efficient inference
Generally Intelligent
Sparse vs Dense Matrices for Different Applications
Sparse matrices can work well for image classification models but may not perform as well for language models with large data sets./nFor applications that can leverage flash transform, like audio or image classification with convolution, sparse matrices like monarch can work well./nFor applications with less structure, such as language models, denser matrices may be needed./nCompanies like Cerebras have found ways to use sparsity throughout training, tailored to their hardware./nCertain cases, like language model inference, are seeing exploration of sparsity./nLanguage model training typically follows a formula of collecting lots of data and training a large transformer model for good performance.
00:00
Transcript
Episode notes
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode