AWS Podcast #753: Amazon Bedrock Mantle and Developing at the Speed of AI
29 snips
Jan 26, 2026 Joe Magerramov, VP and Distinguished Engineer who helped build Amazon Bedrock and Project Mantle, shares lessons from running AI-first development at scale. He discusses 10x code throughput, the infrastructure and testing shifts needed to support rapid commits, and how teams redesign CI/CD, workflows, and roles to keep pace with agent-driven coding.
AI Snips
Chapters
Transcript
Episode notes
Inference Is Scheduling At Scale
- Mantle is an inference engine that treats Bedrock more like a sophisticated scheduler than a simple web service.
- Prioritization, fairness, placement and efficient fleet utilization are core to inference at scale.
Human Accountability Remains Central
- Joe describes his transition from using LLMs for prototypes to trusting them for production-quality code with human oversight.
- He enforces that every committed line of code has a human name attached and a human is accountable.
Calibrate Prompts To A Sweet Spot
- Find the model's maximum supportable request size and tune prompts to that sweet spot for best throughput.
- Remove ambiguity by setting guardrails and constraints just like you would for a junior engineer.