How Monzo runs data-driven AI experimentation
Engineering Enablement by DX
Data-Driven Evaluation Criteria for AI Tools
Fabien outlines Monzo's 10–15 evaluation metrics, including adoption, retention, AI-written code percentage, acceptance rate, and cost.
In this episode of Engineering Enablement, host Laura Tacho talks with Fabien Deshayes, who leads multiple platform engineering teams at Monzo Bank. Fabien explains how Monzo is adopting AI responsibly within a highly regulated industry, balancing innovation with structure, control, and data-driven decision-making.
They discuss how Monzo runs structured AI trials, measures adoption and satisfaction, and uses metrics to guide investment and training. Fabien shares why the company moved from broad rollouts to small, focused cohorts, how they are addressing existing PR review bottlenecks that AI has intensified, and what they have learned from empowering product managers and designers to use AI tools directly.
He also offers insights into budgeting and experimentation, the results Monzo is seeing from AI-assisted engineering, and his outlook on what comes next, from agent orchestration to more seamless collaboration across roles.
Where to find Fabien Deshayes:
• LinkedIn: https://www.linkedin.com/in/fabiendeshayes
Where to find Laura Tacho:
• LinkedIn: https://www.linkedin.com/in/lauratacho/
• Website: https://lauratacho.com/
• Laura’s course (Measuring Engineering Performance and AI Impact): https://lauratacho.com/developer-productivity-metrics-course
In this episode, we cover:
(00:00) Intro
(01:01) An overview of Monzo bank and Fabien’s role
(02:05) Monzo’s careful, structured approach to AI experimentation
(05:30) How Monzo’s AI journey began
(06:26) Why Monzo chose a structured approach to experimentation and what criteria they used
(09:21) How Monzo selected AI tools for experimentation
(11:51) Why individual tool stipends don’t work for large, regulated organizations
(15:32) How Monzo measures the impact of AI tools and uses the data
(18:10) Why Monzo limits AI tool trials to small, focused cohorts
(20:54) The phases of Monzo’s AI rollout and how learnings are shared across the organization
(22:43) What Monzo’s data reveals about AI usage and spending
(24:30) How Monzo balances AI budgeting with innovation
(26:45) Results from DX’s spending poll and general advice on AI budgeting
(28:03) What Monzo’s data shows about AI’s impact on engineering performance
(29:50) The growing bottleneck in PR reviews and how Monzo is solving it with tenancies
(33:54) How product managers and designers are using AI at Monzo
(36:36) Fabien’s advice for moving the needle with AI adoption
(38:42) The biggest changes coming next in AI engineering
Referenced:
The AI-powered Podcast Player
