Why measuring AI impact is difficult

Laura asks why AI impact measurement is hard; Brian, Collin, Ciera, and Eirini outline multidimensional challenges and frameworks like SPACE.

Play episode from 02:01

chevron_right

Transcript

chevron_right

Transcript

Episode notes

As AI adoption accelerates across the software industry, engineering leaders are increasingly focused on a harder question: how to understand whether these tools are actually improving developer experience and organizational outcomes.

In this year-end episode of the Engineering Enablement podcast, host Laura Tacho is joined by Brian Houck from Microsoft, Collin Green and Ciera Jaspan from Google, and Eirini Kalliamvakou from GitHub to examine what 2025 research reveals about AI impact in engineering teams. The panel discusses why measuring AI’s effectiveness is inherently complex, why familiar metrics like lines of code continue to resurface despite their limitations, and how multidimensional frameworks such as SPACE and DORA provide a more accurate view of developer productivity.

The conversation also looks ahead to 2026, exploring how AI is beginning to reshape the role of the developer, how junior engineers’ skill sets may evolve, where agentic workflows are emerging, and why some widely shared AI studies were misunderstood. Together, the panel offers a grounded perspective on moving beyond hype toward more thoughtful, evidence-based AI adoption.

Where to find Brian Houck:

• LinkedIn: https://www.linkedin.com/in/brianhouck/

• Website: https://www.microsoft.com/en-us/research/people/bhouck/

Where to find Collin Green:

• LinkedIn: https://www.linkedin.com/in/collin-green-97720378

• Website: https://research.google/people/107023

Where to find Ciera Jaspan:

• LinkedIn: https://www.linkedin.com/in/ciera

• Website: https://research.google/people/cierajaspan/

Where to find Eirini Kalliamvakou:

• LinkedIn: https://www.linkedin.com/in/eirini-kalliamvakou-1016865/

• X: https://x.com/irina_kAl