Exploring Multi-Query Attention vs Multi-Head Attention in Transformer Architectures

Jon Krohn sits down with Sebastian Raschka to discuss his latest book, Machine Learning Q and AI, the open-source libraries developed by Lightning AI, how to exploit the greatest opportunities for LLM development, and what’s on the horizon for LLMs.

This episode is brought to you by the DataConnect Conference, and by Data Universe, the out-of-this-world data conference. Interested in sponsoring a SuperDataScience Podcast episode? Visit passionfroot.me/superdatascience for sponsorship information.

In this episode you will learn:
• All about Machine Learning Q and AI [04:13]
• Sebastian Raschka’s role as Staff Research Engineer at Lightning AI [19:21]
• PyTorch Lightning’s and Lightning Fabric’s capabilities [39:32]
• Large language models: Opportunities and challenges [43:35]
• DoRA vs LoRA [48:56]
• How to be a successful AI educator [1:34:18]

Additional materials: www.superdatascience.com/767