Episode 77: Open Source Metrics at Microsoft

9 snips

Jan 16, 2024

Guest

James Siri

In this episode, Dawn Foster hosts a discussion with three guests from Microsoft's Open Source Programs Office: Emma Irwin, James Siri, and Justin Gosses. They talk about how Microsoft measures the health of open source communities, their experiences with the CHAOSS Community, and the critical role of open source within the organization. Topics include metrics, tackling security issues, the value of open source contributions, and tracking and improving processes at Microsoft.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Microsoft's Multi-Tier Metrics Approach

Microsoft uses a multi-tiered approach to open source metrics focusing on maintainers, component intelligence, and engineering standards on GitHub.
Metrics are tailored to specific groups and projects, emphasizing motivation and iterative improvement.

INSIGHT

OpenSSF Scorecard Security Experiment

Microsoft experimented with OpenSSF scorecard to motivate repository security improvements.
Although cultural integration of security practices is challenging, some teams align their work with these metrics for management visibility.

INSIGHT

Combining Health and Security Metrics

Combining CHAOSS community health metrics with secure supply chain data helps assess component sustainability.
This integration enables developers to make informed dependency decisions considering security and sustainability risks.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Thank you to the folks at Sustain for providing the hosting account for CHAOSSCast!

CHAOSScast – Episode 77

In this episode of CHAOSScast, host Dawn Foster has a compelling discussion with three guests from Microsoft’s Open Source Programs Office: Emma Irwin, James Siri, and Justin Gosses. The conversation includes how Microsoft measures the health of open source communities, their experiences with the CHAOSS Community, and the critical role of open source within the organization. Topics such as use of metrics, tackling security issues within scaling, and the future of metrics within the company were discussed. Also, they talk about the value of open source contributions within the business, the role of internal communities, and how they track and improve processes at Microsoft, emphasizing the importance of open source impact both externally and internally. Download this episode now to hear more!

[00:00:24] Emma, James, and Justin share their backgrounds with us.

[00:01:53] Emma discusses Microsoft’s multi-tier approach to metrics, focusing on maintainers’ value to products and communities, component intelligence, and engineering standards on GitHub.

[00:04:06] James elaborates on his focus on GitHub metrics, the development of policy and tooling for security, and simplifying developers’ workflow.

[00:04:51] Justin categorizes metrics into those for maintainers, for management, and for developers making decisions on dependencies. He talks about challenges in managing the scale of data from 13,000 repositories and the importance of security metrics.

[00:05:37] Emma discusses an experiment with the OpenSSF scorecard for repository security and the effort to motivate improvements in this area. She highlights the challenges of instilling these practices as part of the culture.

[00:07:30] Justin sees opportunities to combine CHAOSS metrics with secure supply chain efforts, aiming to aid developers in making informed decisions about dependencies and warning them of potential risks.

[00:09:11] Dawn asks about the challenges of scaling metrics and managing the vast number of dependencies. Justin responds by describing an experience focused on aiding developers at the start of a project, helping them make data-informed choices about a few key dependencies.

[00:12:51] Emma adds that from the Open Source Programs Office (OSPO) perspective, having a dashboard to direct inquiries is very helpful. James mentions that the dashboard also provides an easy way to surface security guidance.

[00:13:27] The conversation shifts to Dawn asking about the business aspect of open source within Microsoft and how they measure this impact. James responds that open source is integral to Microsoft’s software development approach, aiming to build an internal community and avoid duplicating solutions. He also discusses the importance of Software Bill of Materials (SBOMs) for security and supply chain transparency.

[00:16:00] Emma elaborates on the internal value of external open source contributions, sharing how they help maintainers demonstrate the business impact during reviews.

[00:17:14] Dawn inquiries about the future direction for Microsoft regarding metrics and measurement. Justin touches on exploring the area of funding, aiming to improve conversations about financial contributions to open source projects and achieving better return on investment.

[00:19:10] James mentions that their package selection work for developers has been inspired by CHAOSS metrics, suggesting that these insights be shared in OSPO working group meetings.

Value Adds (Picks) of the week:

[00:19:34] Dawn’s pick is getting her permanent residency approval allowing her to live in the UK without any restrictions.
[00:19:59] Emma’s pick is taking a break over the holidays and being outside as much as possible.
[00:20:33] Justin’s pick is a book he enjoyed reading called, Elinor Ostrom: An Intellectual Biography.
[00:21:19] James’s pick is reconnecting with art and music as an avenue for self-expression.

*Panelist: *

Dawn Foster

Guests:

Emma Irwin

Justin Gosses

James Siri

Links:

CHAOSS

CHAOSS Project X/Twitter

CHAOSScast Podcast

podcast@chaoss.community

Georg Link Website