How Do We Quantify AI Impact on Productivity?

“Engineering leaders are stuck between the expectations put out by sensational headlines and the reality of what they’re seeing in their organization. There’s a big disappointment gap.”

Is your AI investment paying off? Many leaders struggle to see real ROI beyond the hype.

In this episode, Laura Tacho, CTO of DX, shares DX’s new research on measuring AI adoption success across 38,000+ engineers. Our conversation reveals why acceptance rates are misleading metrics and introduces DX’s new AI Measurement Framework™ with its three critical dimensions: utilization, impact, and cost. Learn why treating AI as an organizational problem closes the “disappointment gap” between hype and reality.

Note: This episode was recorded in July 2025. The AI adoption rate mentioned has since risen to nearly 80%.

In this episode, you will learn about:

• The “Disappointment Gap” between AI hype and reality
• Why the popular “acceptance rate” metric is misleading
• The DX AI Measurement Framework™ and its three dimensions
• The top time-saving AI use case (it’s not code generation!)
• How AI impacts long-term software quality and maintainability
• Why organizational readiness matters for successful AI adoption
• The bigger bottlenecks beyond coding that AI has not yet solved
• Treating AI agents as team extensions, not digital employees

Timestamps:

• (00:00:00) Trailer & Intro
• (00:02:32) Latest DX Research on AI Adoption
• (00:03:54) AI Role on Developer Experience
• (00:05:43) The Current AI Adoption Rate in the Industry
• (00:09:27) The Leader’s Challenges Against Al Hype
• (00:13:22) Measuring AI Adoption ROI Using Acceptance Rate
• (00:17:39) The DX AI Measurement Framework™
• (00:23:05) AI Measurement Framework: Utility Dimension
• (00:27:51) DX AI Code Metrics
• (00:30:31) AI Measurement Framework: Impact Dimension
• (00:32:57) The Importance of Measuring Productivity Holistically
• (00:35:54) AI Measurement Framework: Cost Dimension
• (00:38:34) AI Second Order Impact on Software Quality and Maintainability
• (00:42:38) The Danger of Vibe Coding
• (00:46:31) Treating AI as Extensions of Teams
• (00:52:31) The Bigger Bottlenecks to Solve Outside of AI Adoption
• (00:55:47) DX Guide to AI-Assisted Engineering
• (01:00:38) Being Deliberate for a Successful AI Rollout
• (01:02:32) 3 Tech Lead Wisdom

_____

Laura Tacho’s Bio
Laura Tacho is CTO at DX, a developer intelligence platform, co-author of the Core 4 developer productivity metrics framework, and an executive coach. She’s an experienced technology leader and engineering leadership coach with a strong background in developer tools and distributed systems.

Her career includes leadership roles at organizations such as CloudBees, Aula Education, and Nova Credit, where she specialized in building high-performing engineering teams and delivering impactful products. Laura has worked with thousands of engineering leaders as they work to improve their engineering practices with data.

Follow Laura:

• LinkedIn – linkedin.com/in/lauratacho
• Twitter – x.com/rhein_wein
• Website – lauratacho.com
•  AI Measurement Framework – getdx.com/whitepaper/ai-measurement-framework/?utm_source=techleadjournal
•  Guide to AI-Assisted Engineering – getdx.com/guide/ai-assisted-engineering/?utm_source=techleadjournal
• AI code metrics – getdx.com/ai-code-metrics

Like this episode?
Show notes & transcript: techleadjournal.dev/episodes/233.
Follow @techleadjournal on LinkedIn, Twitter, and Instagram.
Buy me a coffee or become a patron.