Episode 77: Open Source Metrics at Microsoft
CHAOSScast
Approach to Using Metrics at Microsoft
Discussion on the multi-pronged and multi-tiered approach to using metrics within Microsoft, emphasizing the importance of motivation, iteration, and identifying relevant metrics for different groups and projects.
Thank you to the folks at Sustain for providing the hosting account for CHAOSSCast!
CHAOSScast – Episode 77
In this episode of CHAOSScast, host Dawn Foster has a compelling discussion with three guests from Microsoft’s Open Source Programs Office: Emma Irwin, James Siri, and Justin Gosses. The conversation includes how Microsoft measures the health of open source communities, their experiences with the CHAOSS Community, and the critical role of open source within the organization. Topics such as use of metrics, tackling security issues within scaling, and the future of metrics within the company were discussed. Also, they talk about the value of open source contributions within the business, the role of internal communities, and how they track and improve processes at Microsoft, emphasizing the importance of open source impact both externally and internally. Download this episode now to hear more!
[00:00:24] Emma, James, and Justin share their backgrounds with us.
[00:01:53] Emma discusses Microsoft’s multi-tier approach to metrics, focusing on maintainers’ value to products and communities, component intelligence, and engineering standards on GitHub.
[00:04:06] James elaborates on his focus on GitHub metrics, the development of policy and tooling for security, and simplifying developers’ workflow.
[00:04:51] Justin categorizes metrics into those for maintainers, for management, and for developers making decisions on dependencies. He talks about challenges in managing the scale of data from 13,000 repositories and the importance of security metrics.
[00:05:37] Emma discusses an experiment with the OpenSSF scorecard for repository security and the effort to motivate improvements in this area. She highlights the challenges of instilling these practices as part of the culture.
[00:07:30] Justin sees opportunities to combine CHAOSS metrics with secure supply chain efforts, aiming to aid developers in making informed decisions about dependencies and warning them of potential risks.
[00:09:11] Dawn asks about the challenges of scaling metrics and managing the vast number of dependencies. Justin responds by describing an experience focused on aiding developers at the start of a project, helping them make data-informed choices about a few key dependencies.
[00:12:51] Emma adds that from the Open Source Programs Office (OSPO) perspective, having a dashboard to direct inquiries is very helpful. James mentions that the dashboard also provides an easy way to surface security guidance.
[00:13:27] The conversation shifts to Dawn asking about the business aspect of open source within Microsoft and how they measure this impact. James responds that open source is integral to Microsoft’s software development approach, aiming to build an internal community and avoid duplicating solutions. He also discusses the importance of Software Bill of Materials (SBOMs) for security and supply chain transparency.
[00:16:00] Emma elaborates on the internal value of external open source contributions, sharing how they help maintainers demonstrate the business impact during reviews.
[00:17:14] Dawn inquiries about the future direction for Microsoft regarding metrics and measurement. Justin touches on exploring the area of funding, aiming to improve conversations about financial contributions to open source projects and achieving better return on investment.
[00:19:10] James mentions that their package selection work for developers has been inspired by CHAOSS metrics, suggesting that these insights be shared in OSPO working group meetings.
Value Adds (Picks) of the week:
- [00:19:34] Dawn’s pick is getting her permanent residency approval allowing her to live in the UK without any restrictions.
- [00:19:59] Emma’s pick is taking a break over the holidays and being outside as much as possible.
- [00:20:33] Justin’s pick is a book he enjoyed reading called, Elinor Ostrom: An Intellectual Biography.
- [00:21:19] James’s pick is reconnecting with art and music as an avenue for self-expression.
*Panelist: *
Dawn Foster
Guests:
Emma Irwin
Justin Gosses
James Siri
Links:
OSS Project Viability: Compliance + Security
Elinor Ostrom: An Intellectual Biography by Vlad Tarko
Special Guests: Emma Irwin, James Siri, and Justin Gosses.
The AI-powered Podcast Player
