Developing Metrics and Metric Models in the Chaos Project

This chapter discusses the ongoing development of metrics and metric models in the Chaos Project, focusing on the importance of understanding context and the needs of the open source community. It highlights the creation of context groups, the consolidation and release of metrics, as well as the introduction of metric models for analyzing community health. The chapter explores the challenges and initiatives within the data science group of chaos projects, including guidance on interpreting metrics and building a collaborative data science community.

Transcript

chevron_right

Play full episode

chevron_right

Transcript

Episode notes

Thank you to the folks at Sustain for providing the hosting account for CHAOSSCast!

CHAOSScast – Episode 71

In this episode, the CHAOSScast team is back! Georg Link, Dawn Foster, Sean Goggins, Matt Germonprez, and Elizabeth Barron discuss the relaunch of the podcast after taking a short break. They delve into the fascinating world of open source community health, focusing on metrics, metric models, and the CHAOSS Project’s role in measuring the health of open source communities. They share insights on how they’re working to make metrics more accessible and how they interpret these metrics within the context of specific projects. Additionally, they highlight the Data Science Initiative, the growth of CHAOSS community chapters worldwide, and their initiative to improve newcomer experience and promote diversity and inclusion in open source. Download this episode now to find out much more!

[00:02:48] We hear more about where CHAOSS is with developing metrics and metric models and the Context Groups they’ve developed to bring together individuals interested in the health of specific projects or communities.

[00:06:06] The Metric Development Process is brought up, which is the process of defining and releasing metrics has evolved. While some working groups still develop metrics, there’s an effort to consolidate and organize metrics to make them more accessible to users, including categorizing and tagging them.

[00:08:11] Dawn brings up Metrics Models which are collections of metrics that provide insights into specific aspects of open source community health. These models help users understand various phenomena in open source software health and use metrics effectively.

[00:12:14] Georg brings up something new called the Data Science Initiative within CHAOSS, and Dawn talks about her role as Director of Data Science. The initiative aims to provide guidance to users of CHAOSS metrics and tools for interpreting data effectively and she tells us all the key areas that it’s focused on.

[00:16:14] Matt asks Dawn about the balance between maintain an agnostic stance on metrics and providing more guidance to users in interpreting metrics. Dawn discusses the importance of helping users interpret metrics in the context of their specific projects.

[00:17:55] Georg and Dawn talk about using metrics as pointers to prompt users to investigate specific aspects of their communities and projects.

[00:18:53] Elizabeth asks if CHAOSS should play a role in advising users on how to make changes in their communities based on metric insights without adversely affecting other metrics. Dawn shares her thoughts and Sean mentions the experience of CHAOSS members in evaluating different communities and interpreting metrics.

[00:20:34] Georg expresses excitement about the future of CHAOSS and its journey.

[00:21:54] Sean provides an overview of Augur and its evolution over time, including its ability to capture large volumes of data and the development of an API.

[00:24:19] Georg discusses recent developments in Grimoire Lab, including multi tenancy support, scalability improvements, and optimization of data enrichment processes. He also talks about the migration of Grimoire Lab from Elasticsearch to OpenSearch for data storage and visualization, and Sorting Hat, a module within Grimoire Lab for managing identities.

[00:27:40] Dawn asks about the future use of Kibiter, the Kibana fork used in Grimoire Lab, and Georg confirms a full migration to OpenSearch and Open Search Dashboards, indicating that Kibiter may be phased out.

[00:28:52] Matt asks about recent challenges and achievements related to data management and data cleaning in Augur and Grimoire Lab. Sean mentions the importance of data in operationalizing metrics and making them tangible. Georg emphasizes two critical aspects of data quality.

[00:33:32] Elizabeth shares insight into the growth of the CHAOSS community. She discusses the challenges of managing the growing community, and a group CHAOSS is partnering with called “All in” to develop badging for open source projects, addressing scalability challenges.

[00:41:53] Elizabeth talks about the DEI Reflection Project which was crucial in identifying blind spots and improving the CHAOSS community. It led to valuable recommendations, including enhancing the newcomer experience and promoting diversity and inclusion.

Value Adds (Picks) of the week:

[00:44:30] Georg’s pick is living in his new house that he loves.
[00:45:11] Matt’s pick is his cool morning bike rides to his office.
[00:45:44] Dawn’s pick is a warm, sunny vacation she took in Malta.
[00:46:15] Elizabeth’s pick is seeing her granddaughter getting excited to see flowers, birds, mushrooms, and be out in nature.
[00:46:48] Sean’s pick is his daughter, an English PHD student, who published her first academic paper, and has another up for a revise and resubmit.

*Panelists: *

Georg Link

Dawn Foster

Matt Germonprez

Sean Goggins

Elizabeth Barron

Links:

CHAOSS

CHAOSS Mastodon

CHAOSScast Podcast

podcast@chaoss.community

Ford Foundation