Rethinking Metrics in Open Source Projects

This chapter explores the importance of generating data driven insights and rethinking metrics in open source projects. It discusses the correlation between project health metrics and telemetry data, as well as the significance of metrics like stars and forks. The chapter also addresses the impact of new technologies on metrics in open source communities and emphasizes the need for continual evolution of metrics.

Transcript

chevron_right

Play full episode

chevron_right

Transcript

Episode notes

Thank you to the folks at Sustain for providing the hosting account for CHAOSSCast!

CHAOSScast – Episode 75

In this episode, host Georg is joined by CHAOSS members, Sean, Nicole, Matt, Elizabeth, and Dawn. Today, they delve into the CHAOSS Project’s goals for 2024 and beyond, highlighting efforts to establish CHAOSS metrics and models as formal international standards. The conversation covers their potential ISO standardization, strategies to grow the user community, the importance of increasing collaboration within the CHAOSS contributor community, and enhancing software contributions, acknowledging the need to value non-code contributions. They also touch on the importance of community engagement and the utility of hosted software solutions to make CHAOSS tools more accessible. We are all excited to see where this new journey takes us, and we would love for you to be a part of this journey. Hit download now to hear more!

[00:02:57] The first goal discussed is to establish CHAOSS metrics and metrics models as formal international standards. Matt explains the intention to turn CHAOSS metrics into ISO standards and the early stages of this process with the Joint Development Foundation.

[00:04:37] Dawn adds that having ISO standards will lend more legitimacy and visibility to their metrics. Georg differentiates between de facto standards and the goal of achieving de jure international standards.

[00:06:42] The second goal is creating outreach processes and plans to promote CHAOSS and grow the user community. Nicole talks about increasing awareness and visibility of the CHAOSS Project, developing key messages, a marketing plan, and a roadmap for engagement.

[00:10:20] Sean emphasizes the importance of deliberate and consistent communication. Elizabeth looks forward to sharing their outreach strategies with other open source communities. Dawn discusses focusing on user communities to distinguish them from contributors and to support user growth.

[00:12:42] The third goal is to increase collaboration within the CHAOSS contributor community, with Elizabeth noting the importance of focusing on both user and contributor communities. She discusses different ways to enhance software contributions within CHAOSS and highlights the challenge of recognizing and appreciating non-code contributions, which will be a focus area for improvement.

[00:14:37] Elizabeth talks about strengthening visibility and explicit partnerships with other communities and organizations, attending more events to evangelize CHAOSS and attract new community members, encouraging blogging among community members to support outreach and contribution growth, with a shoutout to Gary White’s work at Verizon, and mentorship programs within CHAOSS.

[00:16:41] Sean agrees on the interconnected nature of efforts to foster community engagement. Elizabeth notes that non-code contributors like project managers and community managers bring valuable skills to the community, Nicole echoes the importance of non-code contributions, Matt reflects on the complexity of community engagement.

[00:20:00] Georg appreciates the ‘Chaotic of the Week’ feature for its community-building benefits. He then brings up the topic of the fourth goal which is to provide hosted software as a service for consuming CHAOSS metrics, and Sean discusses the goal of providing hosted solutions to facilitate access to CHAOSS software and metrics, mentioning a survey that highlighted installation difficulties, and highlights the OSS Compass Project.

[00:23:50] Dawn supports the idea of SaaS solutions for the CHAOSS software to allow less technical users to trial and decide on the best fit for their needs, Matt highlights the benefit of SaaS solutions in reducing resource constraints for different types of organizations engaging with CHAOSS metrics and models, and Georg discusses the synergy between providing hosted SaaS offerings and creating outreach processes to grow the CHAOSS user community.

[00:25:15] Georg brings up the fifth goal and that is how do we provide guidance to use it and the goal is to use driven data insights to provide recommendations that help people generate new insights for their communities. Dawn shares her thoughts and mentions collaborating with various context working groups to understand their unique needs and to help interpret metric accordingly, and the creation of ‘insight guides.’

[00:32:38] Dawn mentions the formation of a new data science working group with plans to involve the community in writing insight guides. Georg shares insights from a study on project health metrics and their correlation with usage data.

[00:34:42] The sixth goal is discussed to evolve data policies to accommodate emerging technologies like artificial intelligence, recognizing that this an area not yet fully addressed, but is important for the future.

Value Adds (Picks) of the week:

[00:37:10] Georg’s pick is a research project he did with Sophia Vargas on metrics.
[00:37:42] Dawn’s pick is an article published in ACM, called “Beyond the Repository.”
[00:38:28] Nicole’s pick is the book, Shoe Dog by Phil Knight.
[00:39:29] Sean’s pick is the book, Becoming Human: A Theory of Ontogeny by Michael Tomasello.
[00:40:15] Matt’s pick is his family coming home for the holidays.
[00:40:41] Elizabeth’s pick is reading about this fascinating expedition into the Cyclops Mountains.

Panelists:

Georg Link

Dawn Foster

Matt Germonprez

Sean Goggins

Nicole Huesman

Elizabeth Barron

Links:

CHAOSS

CHAOSS Project X/Twitter

CHAOSScast Podcast

podcast@chaoss.community