Vanishing Gradients Episode 43: Tales from 400+ LLM Deployments: Building Reliable AI Agents in Production
25 snips
Jan 16, 2025 Hugo chats with Alex Strick van Linschoten, a Machine Learning Engineer at ZenML, who has documented over 400 real-world LLM deployments. They discuss the challenges in deploying AI agents, like hallucinations and cascading failures. Alex reveals practical lessons from corporate giants like Anthropic and Klarna, focusing on structured workflows that enhance reliability. He highlights the evolution of LLM capabilities and shares case studies that underscore the importance of prompt engineering and effective error handling in building robust AI systems.
AI Snips
Chapters
Transcript
Episode notes
Diversity and Fundamentals
- Many LLM deployments show a wide variety of use cases and approaches.
- Fundamentals of software engineering and DevOps still matter greatly.
Problem-Driven Approach
- Successful LLM projects are problem-driven, not technology-driven.
- Focus on solving user needs instead of just applying LLMs.
Prompt Engineering Challenges
- Prompt engineering and management are surprisingly complex in production.
- Prompts are brittle, requiring careful tracking and management throughout an application's lifecycle.