MLOps.community

From MVP to Production // AI in Production Conference

Apr 9, 2024
Experts discuss challenges in scaling AI models from MVP to production, emphasizing user feedback for iterative improvements. They highlight the importance of evaluation methods like human annotation, heuristic analysis, and LM-based evaluation. The conversation explores enhancing AI coding assistants with context, navigating challenges in data ingestion and model updating, and the evolution of evaluation sets and data infrastructure in machine learning.
Ask episode
Chapters
Transcript
Episode notes
1
Introduction
00:00 • 2min
2
Transitioning AI Models from Development to Production
01:33 • 20min
3
Enhancing AI Coding Assistants with Context and Guidelines
21:22 • 7min
4
Navigating Challenges in Data Ingestion and Model Updating for Production Models
28:18 • 3min
5
Evolution of Evaluation Sets and Data Infrastructure in Machine Learning
30:54 • 3min
6
Utilizing Observability Tools for User Feedback Analysis and Introducing Internal Bot for Data Pipeline and Model Evaluations
33:38 • 3min