Tri Dao

Tri Dao, Chief Scientist at Together AI and a professor at Princeton, is a pioneer behind Flash Attention and Mamba. He discusses the dramatic 100x drop in inference costs since ChatGPT, driven by hardware-software co-design and memory optimization. Dao predicts Nvidia's dominance will wane in 2-3 years as specialized chips emerge. He also shares insights on AI models improving expert-level productivity and the challenges of generating quality training data for various domains, while envisioning another 10x cost reduction ahead.

Tri Dao, a recent Stanford PhD grad and Chief Scientist at Together AI, discusses his groundbreaking work on FlashAttention-2, enhancing transformer models for faster inference. He explains how FlashAttention improves efficiency by reducing memory usage from quadratic to linear scaling. The conversation also touches on the importance of memory architecture in GPU performance and the balance of traditional techniques with modern AI innovations. Lastly, Tri reflects on the dynamic landscape of AI research and the rise of open-source contributions in the field.

Tri Dao is a PhD student at Stanford, co-advised by Stefano Ermon and Chris Re. He’ll be joining Princeton as an assistant professor next year. He works at the intersection of machine learning and systems, currently focused on efficient training and long-range context.About Generally Intelligent We started Generally Intelligent because we believe that software with human-level intelligence will have a transformative impact on the world. We’re dedicated to ensuring that that impact is a positive one.  We have enough funding to freely pursue our research goals over the next decade, and our backers include Y Combinator, researchers from OpenAI, Astera Institute, and a number of private individuals who care about effective altruism and scientific research.  Our research is focused on agents for digital environments (ex: browser, desktop, documents), using RL, large language models, and self supervised learning. We’re excited about opportunities to use simulated data, network architecture search, and good theoretical understanding of deep learning to make progress on these problems. We take a focused, engineering-driven approach to research.  Learn more about usWebsite: https://generallyintelligent.com/LinkedIn: linkedin.com/company/generallyintelligent/ Twitter: @genintelligent