Measuring AI's Productivity Gains: Velocity and Beyond

Nicole Forsgren created the most widely used frameworks for measuring developer productivity—DORA and SPACE. She wrote the foundational book Accelerate and is about to release her newest book, Frictionless, a practical guide for helping teams move faster in the AI era. She’s currently Senior Director of Developer Intelligence at Google.

We discuss:

1. Why most productivity metrics are a lie

2. Signs that your engineering team could be moving much faster

3. Why AI accelerates coding but developers aren’t speeding up as much as you think

4. AI’s impact on engineers getting into “flow”

5. Her framework for building and scaling a developer experience team

6. The three components of developer experience: flow state, cognitive load, and feedback loops

—

Brought to you by:

Mercury—The art of simplified finances: https://mercury.com/

WorkOS—Modern identity platform for B2B SaaS, free up to 1 million MAUs: https://workos.com/lenny

Coda—The all-in-one collaborative workspace: https://coda.io/lenny

—

Where to find Nicole Forsgren:

• Twitter: https://twitter.com/nicolefv

• LinkedIn: https://www.linkedin.com/in/nicolefv/

• Website: https://nicolefv.com/

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• X: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Introduction to Nicole Forsgren

(05:09) The concept of developer experience (DevEx)

(08:33) Flow state and cognitive load in the age of AI

(12:02) Challenges in measuring productivity with AI

(21:19) The importance of developer experience for business value

(22:20) Common issues and solutions in developer experience

(26:49) Signs your eng team is moving too slow

(29:52) How AI is improving productivity

(33:32) Real examples of productivity improvements

(36:35) Introducing her new book, Frictionless

(43:40) How to get started building a DevEx team

(45:15) The impact of forming developer experience teams

(46:15)  How to measure the impact of DevEx teams

(48:53) Measuring the impact of AI tools on productivity

(55:16) Survey design for developer experience

(57:59) Popular AI tools for developers

(59:08) Bringing a product mindset to DevEx improvements

(01:00:40) AI corner

(01:02:33) Lightning round and final thoughts

—

Referenced:

• How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, GitHub, Google): https://www.lennysnewsletter.com/p/how-to-measure-and-improve-developer

• DORA: https://dora.dev/

• The SPACE framework: A comprehensive guide to developer productivity: https://getdx.com/blog/space-metrics/

• Measuring developer productivity with the DX Core 4: https://getdx.com/research/measuring-developer-productivity-with-the-dx-core-4/

• Gloria Mark’s website: https://gloriamark.com/

• Taking Flight with Copilot: https://dl.acm.org/doi/10.1145/3589996

• DevEx in Action: https://spawn-queue.acm.org/doi/10.1145/3639443

• CodeX: https://openai.com/codex/

• Devin: https://devin.ai/

• Abi Noda on LinkedIn: https://www.linkedin.com/in/abinoda/

• DX is joining Atlassian: https://getdx.com/blog/dx-is-joining-atlassian/

• GitHub Copilot: https://github.com/features/copilot

• Cursor: https://cursor.com/

• The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell

• Gemini Code Assist: https://codeassist.google/

• Claude Code: https://www.claude.com/product/claude-code

• The AI-native startup: 5 products, 7-figure revenue, 100% AI-written code | Dan Shipper (co-founder/CEO of Every): https://www.lennysnewsletter.com/p/inside-every-dan-shipper

• Love Is Blind on Netflix: https://www.netflix.com/title/80996601

• Shrinking on AppleTV+: https://tv.apple.com/us/show/shrinking/umc.cmc.apzybj6eqf6pzccd97kev7bs

• Ninja Creami: https://www.amazon.com/Ninja-NC301-CREAMi-Containers-Bundle/dp/B0BLGR5JPV/

• Jura coffee maker: https://www.amazon.com/Jura-Nordic-Automatic-Coffee-Machine/dp/B0CF65BFZ1/

—

Recommended books:

• Frictionless: https://developerexperiencebook.com/

• DevEx Workbook: https://developerexperiencebook.com/#workbook

• Outlive: The Science and Art of Longevity: https://www.amazon.com/Outlive-Longevity-Peter-Attia-MD/dp/0593236599

• Back Mechanic: https://www.amazon.com/Back-Mechanic-Stuart-McGill-2015-09-30/dp/B01FKSGJYC

• How Big Things Get Done: The Surprising Factors That Determine the Fate of Every Project, from Home Renovations to Space Exploration and Everything in Between: https://www.amazon.com/How-Big-Things-Get-Done/dp/0593239512/

• The Undoing Project: A Friendship That Changed Our Minds: https://www.amazon.com/dp/B01KBM82M4/

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

Lenny may be an investor in the companies discussed.