Shopify’s developer happiness survey | Mark Côté (Shopify)
Sep 27, 2023
auto_awesome
Mark Côté, Leader of the Developer Infrastructure organization at Shopify, shares insights about Shopify's developer survey program, including survey design, participation rates, and data interpretation. They discuss selecting and refining survey questions, analyzing results, and the value of targeted surveys. The podcast also explores the importance of surveys in improving developer productivity and the trust in survey data at Shopify.
Shopify's developer survey program aims to assess developer needs, measure the effects of developer experience initiatives, and serve as a form of marketing and PR.
The survey results help guide Shopify's developer acceleration group and improve engineering health by identifying areas for improvement and assessing the impact of specific initiatives on developer happiness.
To increase survey participation rates, Shopify focuses on fostering a partnership with its user base, regularly sharing results, discussing implemented changes, and expressing appreciation for participants.
Deep dives
Origin and Purpose of the Developer Survey Program
Shopify's bi-annual developer survey program originated in 2018 with the formation of the developer acceleration team. The survey aimed to assess developer needs, measure the effects of developer experience initiatives, and serve as a form of marketing and PR. It has been conducted twice a year since 2019, providing valuable insights to improve developer satisfaction and prioritize pain points.
The Role of Surveys in Improving Developer Productivity
Shopify uses the developer survey results to guide its developer acceleration group and improve engineering health. While the survey is not used to directly measure productivity, it offers valuable guidance and insights. Specific areas examined include overall satisfaction, tooling comparisons, and developer feedback on code quality, maintenance tasks, and other aspects impacting productivity. The data collected helps identify areas for improvement and assess how specific initiatives have impacted developer happiness.
Strategies for Driving Participation Rates
To increase survey participation rates, Shopify has focused on fostering a partnership with its user base. Regularly sharing survey results, discussing implemented changes, and highlighting the impact of developer feedback on improvements helps build trust. Reminders and expressions of appreciation for participants are also important in maintaining engagement. The goal is to ensure developers understand that their input is valued, and the survey is a means to collaboratively enhance their day-to-day experience.
Survey Design and Methodology
Shopify's developer survey design is a collaborative effort involving the developer acceleration team, people analytics, and talent research teams. The process includes formulating clear and consistent questions that capture specific pain points and relevant aspects of developer experience. Iterative improvements are made, balancing the need for continuous data collection with the importance of maintaining comparability over time. The survey is typically around 15 to 20 minutes long and covers areas such as overall satisfaction, tool usage, and pain point ranking.
Interpreting and Utilizing Survey Results
Shopify analyzes survey results by examining pain point rankings and overall satisfaction scores. The ranking of pain points helps identify areas for improvement and measure progress over time. Shopify also considers specific satisfaction scores associated with tools and systems that directly impact the developer experience. This data aids in understanding correlations between satisfaction and critical factors like documentation, CI times, and production infrastructure. Results are shared through reports and presentations, and action plans are developed to address pain points highlighted in the survey.
This week we’re joined by Mark Côté, who leads the Developer Infrastructure organization at Shopify, to learn about their developer survey program. Mark shares what went into designing and running the survey, what they’ve done to drive participation rates higher, and how they interpret their data.
