The Blueprint For AI Agents That Work (ft Diamond Bishop)

Master the art of building AI agents and powerful AI teammates with Diamond Bishop, Director of Engineering and AI at Datadog. In this deep dive, we explore crucial strategies for creating self-improving agent systems, from establishing robust evaluations (evals) and observability to designing effective human-in-the-loop escape hatches. Learn the secrets to building user trust, deciding between prompt engineering and fine-tuning, and managing data sets for peak performance. Diamond shares his expert insights on architecting agents, using LLM as a judge for quality control, and the future of ambient AI in DevSecOps. If you're looking to build your own AI assistant, this episode provides the essential principles and practical advice you need to get started and create systems that learn and improve over time.

Guest: Diamond Bishop, Director of Engineering and AI at Datadog

Learn more about Bits AI SRE: https://www.datadoghq.com/blog/bits-ai-sre/

Datadog MCP Server for Agents: https://www.datadoghq.com/blog/datadog-remote-mcp-server/

Sign up for A.I. coaching for professionals at: https://www.anetic.co

Get FREE AI tools

pip install tool-use-ai

Connect with us

https://x.com/ToolUseAI

https://x.com/MikeBirdTech

https://x.com/diamondbishop

00:00:00 - Intro

00:03:55 - When To Use an Agent vs a Script

00:05:44 - How to Architect an AI Agent

00:08:07 - Prompt Engineering vs Fine-Tuning

00:11:29 - Building Your First Eval Suite

00:26:06 - The Unsolved Problem in Agent Building

00:31:10 - The Future of Local AI Models & Privacy

Subscribe for more insights on AI tools, productivity, and agents.

Tool Use is a weekly conversation with AI experts brought to you by Anetic.