Practical AI Optimizing for efficiency with IBM’s Granite
167 snips
Mar 14, 2025 
Kate Soule, Director of Technical Product Management at IBM's Granite, shares insights on the evolution of AI models. She emphasizes the importance of efficiency over conventional leaderboard scores. The conversation explores breaking tasks into manageable components, co-designing models with hardware, and the innovative 'mixture of experts' approach for speed and efficiency. Soule also discusses the shift towards open-source models, ethical considerations in AI, and the exciting advancements in Granite 3.2, marking a new frontier in performance.
AI Snips
Chapters
Transcript
Episode notes
Kate Soule's Background
- Kate Soule leads product management for Granite, IBM's large language models.
- She joined IBM Research years before LLMs became popular, coming from a business background.
LLMs as Building Blocks
- LLMs are reusable building blocks for various applications, reducing the need for bespoke models.
- IBM centralizes LLM development in research to support diverse use cases and open-source collaboration.
Why Open Source?
- IBM chose Apache 2 license for Granite to promote broad usage and customization.
- Open source fosters stronger software and aligns with IBM's strategy, building upon their experience with Red Hat.