The Transparency of the Developer Satisfaction Survey

4min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

The developer satisfaction survey data is shared transparently within the organization, with aggregated responses being made available to VPs and individual contributors simultaneously. Access to dashboards and aggregated tables allows for DIY analysis by teams. Initially, key stakeholders were targeted to build buy-in for the survey program before broader sharing was initiated. This transparency has led to goodwill and enabled teams to create their own dashboards and analyze data for their specific needs. The structured and open-ended questions in the survey provide valuable insights, especially the detailed responses which help identify specific problem areas and shape product roadmaps. The open text responses have proven to be a valuable source of information for teams lacking user research resources, enabling them to make informed decisions based on developer feedback.

This week we’re joined by Ciera Jaspan and Collin Green, who lead the Engineering Productivity Research team at Google. Ciera and Collin have written several papers from studies they’ve conducted, and this discussion covers the insights from their research as well as their work more broadly at Google.

Discussion points:

(1:19) About the Engineering Productivity Research team
(3:57) How the team interacts with the rest of the organization
(5:58) The different backgrounds included on the team
(13:11) How Google measures developer productivity
(18:54) Evaluating discrepancies between qualitative and quantitative data
(28:40) Google’s quarterly developer survey
(32:02) Distributing survey results back to the organization
(40:25) Misunderstandings about surveys
(43:51) Ciera and Collin’s paper on why measuring productivity is difficult
(50:35) Reductionist metrics for measuring productivity
(55:26) Examples of other fields that have struggled with measurement
(59:00) Google’s study on measuring technical debt
(1:08:05) Human judgment in measurement

Mentions and links:
Follow Ciera and Collin on LinkedIn
A Human-Centered Approach to Measuring Developer Productivity - Paper, Abi’s summary
Enabling the Study of Software Development with Cross-Tool Logs - Paper
Defining, Measuring, and Managing Tech Debt - Paper, Abi’s summary
Google’s Goals, Signals, Metrics framework - Paper, Abi’s summary