Engineering Enablement by DX How Monzo runs data-driven AI experimentation
14 snips
Oct 31, 2025 Fabien Deshayes, platform engineering leader at Monzo Bank, oversees developer experience and AI engineering. He shares insights on Monzo's structured approach to AI experimentation, balancing innovation with regulatory compliance. Fabien discusses the shift from broad rollouts to focused cohorts, leveraging metrics to track adoption and satisfaction. He highlights budgeting strategies for AI and the impact seen in productivity, revealing that AI now contributes to 20% of code and aids in faster prototyping for non-engineers. His outlook on orchestration tools promises exciting developments ahead.
AI Snips
Chapters
Transcript
Episode notes
Run Structured Trials Before Scaling
- Do run structured, data-driven trials before full AI rollout to decide what fits your context.
- Use clear evaluation criteria so you can compare tools objectively and avoid impulse decisions.
Measure Retention, Not Just Signups
- Measuring retention and satisfaction matters more than initial signups to judge real value.
- Track lines of AI-written code, acceptance rates, use-case fit, and cost to measure ROI.
Protect Data With Allow-Listed Knowledge Stores
- Avoid enabling broad data access without guardrails; create allow-listed knowledge stores instead.
- Provide curated Monzo context (markdown docs, RAG) so models produce company-appropriate outputs.