The Cloudcast RAG That Survives Production
44 snips
Jan 14, 2026 Adam Kamor, co-founder and head of engineering at Tonic AI, dives into the latest in AI with a focus on RAG systems. He discusses how organizations can customize large language models, weighing options like RAG against fine-tuning and direct prompts. Adam highlights the challenges RAG faces in its evolution, including operational complexities. He also explains the importance of validating LLM outputs and introduces Tonic Validate as a solution for ensuring data quality and compliance while navigating the delicate balance of privacy and security in sensitive data handling.
AI Snips
Chapters
Transcript
Episode notes
Start Simple: Prompt Before You Build
- Try the simplest customization first: prompt the LLM directly before adding complexity.
- Move to RAG only if prompting fails, then consider fine-tuning as a later, harder step.
RAG's Promise Hits Data Reality
- RAG can deliver mediocre results quickly but requires much more work for production-grade, customer-facing systems.
- Clean data is the critical bottleneck that makes RAG painful in real deployments.
Context Windows Reduce RAG Need
- As context windows grow, prompting covers more use cases and reduces RAG's necessity.
- Many teams now skip RAG and either prompt or invest in fine-tuning for higher-quality, stable results.