CHAOSScast

Episode 77: Open Source Metrics at Microsoft

Jan 16, 2024

In this episode, Dawn Foster hosts a discussion with three guests from Microsoft's Open Source Programs Office: Emma Irwin, James Siri, and Justin Gosses. They talk about how Microsoft measures the health of open source communities, their experiences with the CHAOSS Community, and the critical role of open source within the organization. Topics include metrics, tackling security issues, the value of open source contributions, and tracking and improving processes at Microsoft.

22:41

Creator website

Episode guests

Justin Gosses

Emma Irwin

James Siri

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Microsoft focuses on understanding maintainers' needs, providing component intelligence, and enforcing engineering standards through a multi-pronged approach to metrics.

They leverage chaos metrics models to provide warnings to developers about the sustainability and security risks of dependencies and experiment with the OpenSSF scorecard to motivate maintainers to improve security.

Deep dives

Approach to Metrics at Microsoft

Microsoft takes a multi-pronged approach to metrics, focusing on understanding maintainers' needs, providing component intelligence, and enforcing engineering standards. They aim to understand what brings value to products and communities, co-design solutions, and motivate action.

Introduction

2min

Approach to Using Metrics at Microsoft

4min

Work on Security Metrics and Data at Scale, OpenSSF Scorecard, and Combining Metrics Models

4min

Scaling Metrics and Making Data-Informed Choices

10min

Applying Metrics, Ospo Working Group, and Personal Achievements

4min

Thank you to the folks at Sustain for providing the hosting account for CHAOSSCast!

CHAOSScast – Episode 77

In this episode of CHAOSScast, host Dawn Foster has a compelling discussion with three guests from Microsoft’s Open Source Programs Office: Emma Irwin, James Siri, and Justin Gosses. The conversation includes how Microsoft measures the health of open source communities, their experiences with the CHAOSS Community, and the critical role of open source within the organization. Topics such as use of metrics, tackling security issues within scaling, and the future of metrics within the company were discussed. Also, they talk about the value of open source contributions within the business, the role of internal communities, and how they track and improve processes at Microsoft, emphasizing the importance of open source impact both externally and internally. Download this episode now to hear more!

[00:00:24] Emma, James, and Justin share their backgrounds with us.

[00:01:53] Emma discusses Microsoft’s multi-tier approach to metrics, focusing on maintainers’ value to products and communities, component intelligence, and engineering standards on GitHub.

[00:04:06] James elaborates on his focus on GitHub metrics, the development of policy and tooling for security, and simplifying developers’ workflow.

[00:04:51] Justin categorizes metrics into those for maintainers, for management, and for developers making decisions on dependencies. He talks about challenges in managing the scale of data from 13,000 repositories and the importance of security metrics.

[00:05:37] Emma discusses an experiment with the OpenSSF scorecard for repository security and the effort to motivate improvements in this area. She highlights the challenges of instilling these practices as part of the culture.

[00:07:30] Justin sees opportunities to combine CHAOSS metrics with secure supply chain efforts, aiming to aid developers in making informed decisions about dependencies and warning them of potential risks.

[00:09:11] Dawn asks about the challenges of scaling metrics and managing the vast number of dependencies. Justin responds by describing an experience focused on aiding developers at the start of a project, helping them make data-informed choices about a few key dependencies.

[00:12:51] Emma adds that from the Open Source Programs Office (OSPO) perspective, having a dashboard to direct inquiries is very helpful. James mentions that the dashboard also provides an easy way to surface security guidance.

[00:13:27] The conversation shifts to Dawn asking about the business aspect of open source within Microsoft and how they measure this impact. James responds that open source is integral to Microsoft’s software development approach, aiming to build an internal community and avoid duplicating solutions. He also discusses the importance of Software Bill of Materials (SBOMs) for security and supply chain transparency.

[00:16:00] Emma elaborates on the internal value of external open source contributions, sharing how they help maintainers demonstrate the business impact during reviews.

[00:17:14] Dawn inquiries about the future direction for Microsoft regarding metrics and measurement. Justin touches on exploring the area of funding, aiming to improve conversations about financial contributions to open source projects and achieving better return on investment.

[00:19:10] James mentions that their package selection work for developers has been inspired by CHAOSS metrics, suggesting that these insights be shared in OSPO working group meetings.

Value Adds (Picks) of the week:

[00:19:34] Dawn’s pick is getting her permanent residency approval allowing her to live in the UK without any restrictions.
[00:19:59] Emma’s pick is taking a break over the holidays and being outside as much as possible.
[00:20:33] Justin’s pick is a book he enjoyed reading called, Elinor Ostrom: An Intellectual Biography.
[00:21:19] James’s pick is reconnecting with art and music as an avenue for self-expression.

*Panelist: *

Dawn Foster

Guests:

Emma Irwin

Justin Gosses

James Siri

Links:

CHAOSS

CHAOSS Project X/Twitter

CHAOSScast Podcast

podcast@chaoss.community

Georg Link Website