John Berryman (Arcturus Labs; early GitHub Copilot engineer; co-author of Relevant Search and Prompt Engineering for LLMs) has spent years figuring out what makes AI applications actually work in production. In this episode, he shares the “seven deadly sins” of LLM development — and the practical fixes that keep projects from stalling.
From context management to retrieval debugging, John explains the patterns he’s seen succeed, the mistakes to avoid, and why it helps to think of an LLM as an “AI intern” rather than an all-knowing oracle.
We talk through:
- Why chasing perfect accuracy is a dead end
- How to use agents without losing control
- Context engineering: fitting the right information in the window
- Starting simple instead of over-orchestrating
- Separating retrieval from generation in RAG
- Splitting complex extractions into smaller checks
- Knowing when frameworks help — and when they slow you down
A practical guide to avoiding the common traps of LLM development and building systems that actually hold up in production.
LINKS:
🎓 Learn more:
