Engineering Enablement by Abi Noda DORA’s latest research on AI impact
31 snips
May 23, 2025 Derek DeBellis, lead researcher at Google’s DORA team, discusses the intriguing findings from their latest research on generative AI's impact on software productivity. He reveals how language precision in surveys can shift results and emphasizes the importance of measuring 'flow' in development tasks. The conversation addresses a significant 2.1% rise in individual productivity alongside concerns over team efficiency decline. Derek also provides actionable insights for leaders on aligning AI metrics with organizational objectives, stressing the need for nuanced understanding in these evaluations.
AI Snips
Chapters
Books
Transcript
Episode notes
Survey Wording Shapes Validity
- Iterative question development uncovers subtle wording differences that profoundly impact survey responses.
- Even small word changes affect comprehension and internal validity of developer surveys.
Measure Flow with Broad Questions
- Asking about "flow" generally lets respondents apply their own understanding, improving valid measurement.
- Flow is multifaceted; a broad approach captures this complexity in developer surveys.
Relative Accuracy in Time Estimates
- Self-reported time spent on valuable work may be inaccurate in absolute terms but reliable relatively.
- Relative estimates effectively indicate differences in valuable work time among individuals.
