Tool Use - AI Conversations The Blueprint For AI Agents That Work (ft Diamond Bishop)
Jun 24, 2025
Join Diamond Bishop, the Director of Engineering and AI at Datadog, as he shares his expertise in building self-improving AI agents. He discusses strategies for establishing robust evaluation systems, fostering user trust, and choosing between prompt engineering and fine-tuning. Diamond also explores the balance of AI agents versus traditional scripts, the importance of dataset management, and the exciting future of ambient AI in DevSecOps. This conversation is packed with essential insights for anyone looking to create effective, adaptive AI systems.
AI Snips
Chapters
Transcript
Episode notes
Human-in-the-Loop for Trust
- Build human-in-the-loop escape hatches to ensure safety in high-stakes AI agent actions.
- Always keep a human able to verify critical decisions to build user trust and prevent system failures.
When To Use AI Agents
- Use deterministic code if a task can be solved that way to avoid unnecessary complexity.
- Choose AI agents when workflows involve ambiguity, long tails, or require flexible reasoning.
Prioritize Observability & Logging
- Prioritize observability and thorough logging when building AI agents.
- Good debugging and replayability are more critical than framework choice for early success.