Lenny's Podcast: Product | Growth | Career

How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, GitHub, Google)

Jul 30, 2023

Nicole Forsgren, a developer productivity and DevOps expert currently at Microsoft Research, digs into the intricacies of measuring developer productivity. She discusses frameworks like DORA and SPACE, shedding light on the benchmarks of excellence and frequent pitfalls to avoid. Nicole emphasizes the importance of a healthy developer experience, and her Four-Box framework provides a unique approach to analyzing data relationships. Listeners can expect actionable strategies to enhance team culture and productivity, all while embracing the evolving tech landscape.

01:16:17

Creator website

Episode guests

Nicole Forsgren

AI Summary

Highlights

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

The Four Box Framework is a tool for measuring and testing ideas, helping clarify what to measure and how to gather relevant data for analysis.

The DORA Framework with its four key metrics (lead time, deployment frequency, mean time to restore, and change failure rate) can enhance team productivity and developer experience by identifying areas for improvement.

The increasing use of AI in engineering tools presents both opportunities and challenges, with considerations around accuracy, cognitive load, and overall productivity measurement.

Deep dives

The Four Box Framework

The Four Box Framework is a tool for measuring and testing ideas. It consists of four boxes: the words box, the data box, an arrow connecting them, and the data box below. Start with words by defining a hypothesis or idea, such as customer satisfaction leading to return customers. Then, gather stakeholders and validate the idea. Once agreed upon, move to the data boxes and determine how to measure customer satisfaction, such as through surveys or CSAT scores. Repeat the same process for measuring return customers. The Four Box Framework helps clarify what to measure and how to gather relevant data for analysis.

The Metrics that Drive Software Delivery Performance

02:40

Why the Retail Industry's Survival Depends on Performance and Cloud Solutions

01:22

Recommendations for understanding metrics and data from an engineering perspective

01:22

The Framework of SPACE: Measuring Productivity and Complex Creative Work

03:42

Intro

3min

A Journey Through Developer Productivity and Experience

5min

Enhancing Developer Productivity and Experience

21min

Enhancing Developer Productivity through DORA and SPACE Frameworks

3min

Measuring Developer Productivity

9min

Navigating the Challenges of Developer Productivity Measurement

4min

Enhancing Developer Productivity

26min

Entertainment, Insights, and Product Innovation

6min

This episode is brought to you by DX—a platform for measuring and improving developer productivity.

—

Dr. Nicole Forsgren is a developer productivity and DevOps expert who works with engineering organizations to make work better. Best known as co-author of the Shingo Publication Award-winning book Accelerate and the DevOps Handbook, 2nd edition and author of the State of DevOps Reports, she has helped some of the biggest companies in the world transform their culture, processes, tech, and architecture. Nicole is currently a Partner at Microsoft Research, leading developer productivity research and strategy, and a technical founder/CEO with a successful exit to Google. In a previous life, she was a software engineer, sysadmin, hardware performance engineer, and professor. She has published several peer-reviewed journal papers, has been awarded public and private research grants (funders include NASA and the NSF), and has been featured in the Wall Street Journal, Forbes, Computerworld, and InformationWeek. In today’s podcast, we discuss:

• Two frameworks for measuring developer productivity: DORA and SPACE

• Benchmarks for what good and great look like

• Common mistakes to avoid when measuring developer productivity

• Resources and tools for improving your metrics

• Signs your developer experience needs attention

• How to improve your developer experience

• Nicole’s Four-Box framework for thinking about data and relationships

—

Find the full transcript at: https://www.lennysnewsletter.com/p/how-to-measure-and-improve-developer

—

Where to find Nicole Forsgren:

• Twitter: https://twitter.com/nicolefv

• LinkedIn: https://www.linkedin.com/in/nicolefv/

• Website: https://nicolefv.com/

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• Twitter: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Nicole’s background

(07:55) Unpacking the terms “developer productivity,” “developer experience,” and “DevOps”

(10:06) How to move faster and improve practices across the board

(13:43) The DORA framework

(18:54) Benchmarks for success

(22:33) Why company size doesn’t matter

(24:54) How to improve DevOps capabilities by working backward

(29:23) The SPACE framework and choosing metrics

(32:51) How SPACE and DORA work together

(35:39) Measuring satisfaction

(37:52) Resources and tools for optimizing metrics

(41:29) Nicole’s current book project

(45:43) Common pitfalls companies run into when rolling out developer productivity/optimizations

(47:42) How the DevOps space has progressed

(50:07) The impact of AI on the developer experience and productivity

(54:04) First steps to take if you’re trying to improve the developer experience

(55:15) Why Google is an example of a company implementing DevOps solutions well

(56:11) The importance of clear communication

(57:32) Nicole’s Four-Box framework

(1:05:15) Advice on making decisions

(1:08:56) Lightning round

—

Referenced:

• Chef: https://www.chef.io/

• DORA: https://dora.dev/

• GitHub: https://github.com/

• Microsoft Research: https://www.microsoft.com/en-us/research/

• What is DORA?: https://devops.com/what-is-dora-and-why-you-should-care/

• Dustin Smith on LinkedIn: https://www.linkedin.com/in/dustin-smith-b0525458/

• Nathen Harvey on LinkedIn: https://www.linkedin.com/in/nathen/

• What is CI/CD?: https://about.gitlab.com/topics/ci-cd/

• Trunk-based development: https://cloud.google.com/architecture/devops/devops-tech-trunk-based-development

• DORA DevOps Quick Check: https://dora.dev/quickcheck/

• Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations: https://www.amazon.com/Accelerate-Software-Performing-Technology-Organizations/dp/1942788339

• The SPACE of Developer Productivity: https://queue.acm.org/detail.cfm?id=3454124

• DevOps Metrics: Nicole Forsgren and Mik Kersten: https://queue.acm.org/detail.cfm?id=3182626

• How to Measure Anything: Finding the Value of Intangibles in Business: https://www.amazon.com/How-Measure-Anything-Intangibles-Business/dp/1118539273/

• GitHub Copilot: https://github.com/features/copilot

• Tabnine: https://www.tabnine.com/the-leading-ai-assistant-for-software-development

• Nicole’s Decision-Making Spreadsheet: https://docs.google.com/spreadsheets/d/1wItAODkhZ-zKnnFbyDERCd8Hq2NQ03WPvCfigBQ5vpc/edit?usp=sharing

• How to do linear regression and correlation analysis: https://www.lennysnewsletter.com/p/linear-regression-and-correlation-analysis

• Good Strategy/Bad Strategy: The difference and why it matters: https://www.amazon.com/Good-Strategy-Bad-difference-matters/dp/1781256179/

• Designing Your Life: How to Build a Well-Lived, Joyful Life: https://www.amazon.com/Designing-Your-Life-Well-Lived-Joyful/dp/1101875321

• Ender’s Game: https://www.amazon.com/Enders-Game-Ender-Quintet-1/dp/1250773024/ref=tmm_pap_swatch_0

• Suits on Netflix: https://www.netflix.com/title/70195800

• Ted Lasso on AppleTV+: https://tv.apple.com/us/show/ted-lasso

• Never Have I Ever on Netflix: https://www.netflix.com/title/80179190

• Eight Sleep: https://www.eightsleep.com/

• COSRX face masks: https://www.amazon.com/COSRX-Advanced-Secretion-Hydrating-Moisturizing/dp/B08JSL9W6K/

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

—

Lenny may be an investor in the companies discussed.

Get full access to Lenny's Newsletter at www.lennysnewsletter.com/subscribe

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.

Lenny's Podcast: Product | Growth | Career

How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, GitHub, Google)

Episode guests

Podcast summary created with Snipd AI

Quick takeaways

Deep dives

The Four Box Framework

Using the DORA Framework

Understanding the Impact of AI on Engineering Productivity

The SPACE framework for metrics selection

Tips for decision-making and product development

The Metrics that Drive Software Delivery Performance

Why the Retail Industry's Survival Depends on Performance and Cloud Solutions

Recommendations for understanding metrics and data from an engineering perspective

The Framework of SPACE: Measuring Productivity and Complex Creative Work

Remember Everything You Learn from Podcasts