How to Measure Anything and Gather Insights: Surveying Developers and Utilizing Data

This episode is brought to you by DX—a platform for measuring and improving developer productivity.

—

Dr. Nicole Forsgren is a developer productivity and DevOps expert who works with engineering organizations to make work better. Best known as co-author of the Shingo Publication Award-winning book Accelerate and the DevOps Handbook, 2nd edition and author of the State of DevOps Reports, she has helped some of the biggest companies in the world transform their culture, processes, tech, and architecture. Nicole is currently a Partner at Microsoft Research, leading developer productivity research and strategy, and a technical founder/CEO with a successful exit to Google. In a previous life, she was a software engineer, sysadmin, hardware performance engineer, and professor. She has published several peer-reviewed journal papers, has been awarded public and private research grants (funders include NASA and the NSF), and has been featured in the Wall Street Journal, Forbes, Computerworld, and InformationWeek. In today’s podcast, we discuss:

• Two frameworks for measuring developer productivity: DORA and SPACE

• Benchmarks for what good and great look like

• Common mistakes to avoid when measuring developer productivity

• Resources and tools for improving your metrics

• Signs your developer experience needs attention

• How to improve your developer experience

• Nicole’s Four-Box framework for thinking about data and relationships

—

Find the full transcript at: https://www.lennysnewsletter.com/p/how-to-measure-and-improve-developer

—

Where to find Nicole Forsgren:

• Twitter: https://twitter.com/nicolefv

• LinkedIn: https://www.linkedin.com/in/nicolefv/

• Website: https://nicolefv.com/

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• Twitter: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Nicole’s background

(07:55) Unpacking the terms “developer productivity,” “developer experience,” and “DevOps”

(10:06) How to move faster and improve practices across the board

(13:43) The DORA framework

(18:54) Benchmarks for success

(22:33) Why company size doesn’t matter 

(24:54) How to improve DevOps capabilities by working backward

(29:23) The SPACE framework and choosing metrics

(32:51) How SPACE and DORA work together

(35:39) Measuring satisfaction

(37:52) Resources and tools for optimizing metrics

(41:29) Nicole’s current book project

(45:43) Common pitfalls companies run into when rolling out developer productivity/optimizations

(47:42) How the DevOps space has progressed

(50:07) The impact of AI on the developer experience and productivity

(54:04) First steps to take if you’re trying to improve the developer experience

(55:15) Why Google is an example of a company implementing DevOps solutions well

(56:11) The importance of clear communication

(57:32) Nicole’s Four-Box framework

(1:05:15) Advice on making decisions 

(1:08:56) Lightning round

—

Referenced:

• Chef: https://www.chef.io/

• DORA: https://dora.dev/

• GitHub: https://github.com/

• Microsoft Research: https://www.microsoft.com/en-us/research/

• What is DORA?: https://devops.com/what-is-dora-and-why-you-should-care/

• Dustin Smith on LinkedIn: https://www.linkedin.com/in/dustin-smith-b0525458/

• Nathen Harvey on LinkedIn: https://www.linkedin.com/in/nathen/

• What is CI/CD?: https://about.gitlab.com/topics/ci-cd/

• Trunk-based development: https://cloud.google.com/architecture/devops/devops-tech-trunk-based-development

• DORA DevOps Quick Check: https://dora.dev/quickcheck/

• Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations: https://www.amazon.com/Accelerate-Software-Performing-Technology-Organizations/dp/1942788339

• The SPACE of Developer Productivity: https://queue.acm.org/detail.cfm?id=3454124

• DevOps Metrics: Nicole Forsgren and Mik Kersten: https://queue.acm.org/detail.cfm?id=3182626

• How to Measure Anything: Finding the Value of Intangibles in Business: https://www.amazon.com/How-Measure-Anything-Intangibles-Business/dp/1118539273/

• GitHub Copilot: https://github.com/features/copilot

• Tabnine: https://www.tabnine.com/the-leading-ai-assistant-for-software-development

• Nicole’s Decision-Making Spreadsheet: https://docs.google.com/spreadsheets/d/1wItAODkhZ-zKnnFbyDERCd8Hq2NQ03WPvCfigBQ5vpc/edit?usp=sharing

• How to do linear regression and correlation analysis: https://www.lennysnewsletter.com/p/linear-regression-and-correlation-analysis

• Good Strategy/Bad Strategy: The difference and why it matters: https://www.amazon.com/Good-Strategy-Bad-difference-matters/dp/1781256179/

• Designing Your Life: How to Build a Well-Lived, Joyful Life: https://www.amazon.com/Designing-Your-Life-Well-Lived-Joyful/dp/1101875321

• Ender’s Game: https://www.amazon.com/Enders-Game-Ender-Quintet-1/dp/1250773024/ref=tmm_pap_swatch_0

• Suits on Netflix: https://www.netflix.com/title/70195800

• Ted Lasso on AppleTV+: https://tv.apple.com/us/show/ted-lasso

• Never Have I Ever on Netflix: https://www.netflix.com/title/80179190

• Eight Sleep: https://www.eightsleep.com/

• COSRX face masks: https://www.amazon.com/COSRX-Advanced-Secretion-Hydrating-Moisturizing/dp/B08JSL9W6K/

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

—

Lenny may be an investor in the companies discussed.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.lennysnewsletter.com/subscribe