RunAs Radio Managing AI Costs with Sonia Cuff
Mar 19, 2025
In this engaging conversation, Sonja Cuff, a Senior Cloud Ops Advocate on the Azure engineering team, shares her expertise in managing AI costs. She clarifies that AI encompasses more than just large language models. The discussion dives into practical strategies for measuring AI expenditures and improving sustainability using FinOps models. Topics like aligning IT spending with business benefits and the intricacies of app development optimization are explored, while the significance of realistic expectations in the evolving AI landscape is emphasized.
AI Snips
Chapters
Books
Transcript
Episode notes
AI Workloads Are Just Workloads
- AI workloads are just another IT workload requiring security and cost management like any other application.
- Immediate concerns include data flow, security, and understanding billing models to manage AI costs effectively.
GitHub Copilot Productivity Boost
- Richard Campbell observed a 20-30% productivity increase in developers using GitHub Copilot.
- The cost per developer was minimal compared to the productivity gains experienced over time.
Know Your AI Cost Models
- Understand the billing units for AI services; generative AI often charges per token, while others use hourly or SKU-based rates.
- Know where costs come from, as some AI components incur charges continuously, like fine-tuning models billing hourly when deployed.
