Lenny's Podcast: Product | Growth | Career

How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, GitHub, Google)

Jul 30, 2023

01:16:17

Snipd AI

Nicole Forsgren, developer productivity expert, discusses measuring and improving developer productivity. Topics include DORA and SPACE frameworks, benchmarks for good productivity, common measuring mistakes, improving metrics, signs of developer experience issues, and how to improve developer experience using Nicole's Four-Box framework.

AI Summary

Highlights

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

The Four Box Framework is a tool for measuring and testing ideas, helping clarify what to measure and how to gather relevant data for analysis.

The DORA Framework with its four key metrics (lead time, deployment frequency, mean time to restore, and change failure rate) can enhance team productivity and developer experience by identifying areas for improvement.

The increasing use of AI in engineering tools presents both opportunities and challenges, with considerations around accuracy, cognitive load, and overall productivity measurement.

Deep dives

The Four Box Framework

The Four Box Framework is a tool for measuring and testing ideas. It consists of four boxes: the words box, the data box, an arrow connecting them, and the data box below. Start with words by defining a hypothesis or idea, such as customer satisfaction leading to return customers. Then, gather stakeholders and validate the idea. Once agreed upon, move to the data boxes and determine how to measure customer satisfaction, such as through surveys or CSAT scores. Repeat the same process for measuring return customers. The Four Box Framework helps clarify what to measure and how to gather relevant data for analysis.

In Software Development Quality Comes From Shipping Faster

00:43

Measuring Productivity: The Importance of Space

01:22

Using a Spreadsheet to Make Decisions

02:02

Introduction

3min

A Unique Background and Experiences in Software Engineering and Management

5min

Developer Productivity and Developer Experience in DevOps

15min

Benchmarking Developer Productivity Across Different Companies

18min

Measuring and Improving Developer Productivity with DX

4min

Measuring and Improving Developer Productivity

24min

Creating a Public Template and Lightning Round

8min

This episode is brought to you by DX—a platform for measuring and improving developer productivity.

—

Dr. Nicole Forsgren is a developer productivity and DevOps expert who works with engineering organizations to make work better. Best known as co-author of the Shingo Publication Award-winning book Accelerate and the DevOps Handbook, 2nd edition and author of the State of DevOps Reports, she has helped some of the biggest companies in the world transform their culture, processes, tech, and architecture. Nicole is currently a Partner at Microsoft Research, leading developer productivity research and strategy, and a technical founder/CEO with a successful exit to Google. In a previous life, she was a software engineer, sysadmin, hardware performance engineer, and professor. She has published several peer-reviewed journal papers, has been awarded public and private research grants (funders include NASA and the NSF), and has been featured in the Wall Street Journal, Forbes, Computerworld, and InformationWeek. In today’s podcast, we discuss:

• Two frameworks for measuring developer productivity: DORA and SPACE

• Benchmarks for what good and great look like

• Common mistakes to avoid when measuring developer productivity

• Resources and tools for improving your metrics

• Signs your developer experience needs attention

• How to improve your developer experience

• Nicole’s Four-Box framework for thinking about data and relationships

—

Find the full transcript at: https://www.lennysnewsletter.com/p/how-to-measure-and-improve-developer

—

Where to find Nicole Forsgren:

• Twitter: https://twitter.com/nicolefv

• LinkedIn: https://www.linkedin.com/in/nicolefv/

• Website: https://nicolefv.com/

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• Twitter: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Nicole’s background

(07:55) Unpacking the terms “developer productivity,” “developer experience,” and “DevOps”

(10:06) How to move faster and improve practices across the board

(13:43) The DORA framework

(18:54) Benchmarks for success

(22:33) Why company size doesn’t matter

(24:54) How to improve DevOps capabilities by working backward

(29:23) The SPACE framework and choosing metrics

(32:51) How SPACE and DORA work together

(35:39) Measuring satisfaction

(37:52) Resources and tools for optimizing metrics

(41:29) Nicole’s current book project

(45:43) Common pitfalls companies run into when rolling out developer productivity/optimizations

(47:42) How the DevOps space has progressed

(50:07) The impact of AI on the developer experience and productivity

(54:04) First steps to take if you’re trying to improve the developer experience

(55:15) Why Google is an example of a company implementing DevOps solutions well

(56:11) The importance of clear communication

(57:32) Nicole’s Four-Box framework

(1:05:15) Advice on making decisions

(1:08:56) Lightning round

—

Referenced:

• Chef: https://www.chef.io/

• DORA: https://dora.dev/

• GitHub: https://github.com/

• Microsoft Research: https://www.microsoft.com/en-us/research/

• What is DORA?: https://devops.com/what-is-dora-and-why-you-should-care/

• Dustin Smith on LinkedIn: https://www.linkedin.com/in/dustin-smith-b0525458/

• Nathen Harvey on LinkedIn: https://www.linkedin.com/in/nathen/

• What is CI/CD?: https://about.gitlab.com/topics/ci-cd/

• Trunk-based development: https://cloud.google.com/architecture/devops/devops-tech-trunk-based-development

• DORA DevOps Quick Check: https://dora.dev/quickcheck/

• Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations: https://www.amazon.com/Accelerate-Software-Performing-Technology-Organizations/dp/1942788339

• The SPACE of Developer Productivity: https://queue.acm.org/detail.cfm?id=3454124

• DevOps Metrics: Nicole Forsgren and Mik Kersten: https://queue.acm.org/detail.cfm?id=3182626

• How to Measure Anything: Finding the Value of Intangibles in Business: https://www.amazon.com/How-Measure-Anything-Intangibles-Business/dp/1118539273/

• GitHub Copilot: https://github.com/features/copilot

• Tabnine: https://www.tabnine.com/the-leading-ai-assistant-for-software-development

• Nicole’s Decision-Making Spreadsheet: https://docs.google.com/spreadsheets/d/1wItAODkhZ-zKnnFbyDERCd8Hq2NQ03WPvCfigBQ5vpc/edit?usp=sharing

• How to do linear regression and correlation analysis: https://www.lennysnewsletter.com/p/linear-regression-and-correlation-analysis

• Good Strategy/Bad Strategy: The difference and why it matters: https://www.amazon.com/Good-Strategy-Bad-difference-matters/dp/1781256179/

• Designing Your Life: How to Build a Well-Lived, Joyful Life: https://www.amazon.com/Designing-Your-Life-Well-Lived-Joyful/dp/1101875321

• Ender’s Game: https://www.amazon.com/Enders-Game-Ender-Quintet-1/dp/1250773024/ref=tmm_pap_swatch_0

• Suits on Netflix: https://www.netflix.com/title/70195800

• Ted Lasso on AppleTV+: https://tv.apple.com/us/show/ted-lasso

• Never Have I Ever on Netflix: https://www.netflix.com/title/80179190

• Eight Sleep: https://www.eightsleep.com/

• COSRX face masks: https://www.amazon.com/COSRX-Advanced-Secretion-Hydrating-Moisturizing/dp/B08JSL9W6K/

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

—

Lenny may be an investor in the companies discussed.

Get full access to Lenny's Newsletter at www.lennysnewsletter.com/subscribe

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export