S5E2: Why Most Experimentation Programs Fail featuring Manuel Da Costa

Jul 26, 2025

In this conversation, Manuel Da Costa, founder of Effective Experiments and Efestra, shares his 15 years of expertise in the experimentation field. He highlights the crucial 'Trust Gap' between practitioners and decision-makers, emphasizing that running more experiments isn't the goal—making impactful decisions is. Manuel introduces the Learning Loop concept and offers strategies to bridge this divide. He argues for the need of experimentation leaders to be heard as strategic advisors, advocating for meaningful insights that drive real business transformations.

Ask episode

AI Snips

Chapters

Books

Transcript

Episode notes

INSIGHT

Trust Gap Limits Experimentation Impact

Most experimentation programs fail to be transformative due to a "trust gap" between practitioners and executives.
This gap causes insights to be ignored, keeping experimentation teams in tactical roles instead of strategic advisors.

INSIGHT

Survival Mindset Traps Specialists

Experimentation advice often traps practitioners in a survival mindset focused on politics and proving worth.
This mindset prevents teams from evolving into strategic advisors who can influence top-level business decisions.

INSIGHT

Tool Marketing Skews Experimentation Focus

The way experimentation tools market themselves focuses on volume of tests, which shapes how organizations view experimentation.
This emphasis on quantity over learning contributes to undervaluing the insights that lead to better decisions.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

In this episode of the Experiment Nation Podcast, host Richard Joe welcomes Manuel Da Costa, founder of Effective Experiments and now Efestra, to unpack a critical issue in the experimentation industry: the trust gap between practitioners and business decision-makers.🎯 Manuel shares how running more experiments isn’t the goal—making better decisions is. He introduces the Learning Loop, explains why most experimentation programs stall, and gives practical strategies for connecting experimentation insights to executive decisions.🔍 Whether you’re in CRO, product experimentation, or experimentation governance, this is a must-watch for moving beyond vanity metrics and toward real impact.📌 Connect with Manuel:📍 LinkedIn: https://www.linkedin.com/in/manueldacosta/#CRO #Experimentation #ABTesting #EffectiveExperiments #ProductManagement #DigitalMarketingChapter00:00 – Experiments don't matter unless they drive decisions 00:30 – Meet Manuel Da Costa, founder of Effective Experiments 01:45 – Manuel’s journey: from web design to CRO and Lean Startup 03:00 – How experimentation became his product 05:20 – The limitations of spreadsheets in experimentation 06:45 – Building Effective Experiments to streamline the workflow 09:10 – What is the "Trust Gap" in experimentation? 11:30 – Why 88% of experimentation programs aren’t transformative 13:00 – The survival mindset vs. strategic experimentation 16:00 – The echo chamber of experimentation advice 17:40 – Why velocity and win rate are not the real success metrics 19:00 – The problem with how learnings are shared 21:00 – How to communicate insights in one powerful slide 23:00 – Building organizational learning with the “Learning Loop” 25:00 – Step-by-step breakdown of the Learning Loop framework 27:15 – Why experimentation needs executive alignment 29:00 – How to reset a tactical experimentation program 30:40 – Introducing Efestra: a governance platform for experimentation 32:00 – Final thoughts and how to get in touch with Manuel 33:14 – About the book: *Prove It or Lose It*