Exploring Representation, Data Collection, and Biases in Open Source Research

Get the app

Episode 209: Amanda Casari, Julie Ferraioli & Juniper Lovato and open source ecosystems research

Sustain

chevron_right

notes

CHAPTER

Exploring Representation, Data Collection, and Biases in Open Source Research

This chapter explores the challenges of capturing information without risking people's identities, the need to protect vulnerable communities, and the importance of including them in the research process. It also discusses the complexities of identity within the open source context and the biases in data collection.

00:00

Transcript

chevron_right

Play full episode

chevron_right

Transcript

Episode notes

Guests

Amanda Casari | Julie Ferraioli | Juniper Lovato

Panelist

Richard Littauer

Show Notes

In today’s episode of Sustain, Richard is joined by guests, Amanda Casari, devrel engineer and open source researcher at Google Open Source Programs Office, Julie Ferraioli, an independent open source strategist, researcher, practitioner, and Partner at Open Chapters, and Juniper Lovato, Director of partnerships and programs at the Vermont Complex Systems Center at UVM and Data Ethics researcher. Amanda, Julia, and Juniper join the discussion, bringing a wealth of expertise in the open source domain. The conversation gravitates towards an article co-authored by the guests, striking a balance between open source software and open source ecosystems research. The episode dives deep into the “10 simple things” format of the article, the crucial importance of collective conversations, and a keen exploration of open source researchers. Hit download now to hear more cool stuff!

[00:01:29] Richard tells us why he invited our three guests today and he talks about their previous accomplishments and backgrounds.

[00:02:17] Our discussion moves to the title of a new article co-authored by the guests. We hear about the intended audience of the article and the distinction made between open source software and open source ecosystems research.

[00:03:31] Richard brings up where the article fits in the academic landscape, and it’s revealed to be more editorial than research.

[00:04:17] There’s a conversation about the “10 simple things” format, its origin, and the motivation behind it. They put an emphasis on the need for collective conversation and the value of sharing experiences and knowledge.

[00:07:28] Richard brings up the idea of open source researchers and mentions various figures and institutions involved in open source research. Juniper clarifies the target audience for the article and its intentions, Julie shares her perspective from the industry side and the importance of a critical framework, and Amanda expresses her emotional response to some researchers’ approach towards the open source community.

[00:12:03] Julie discusses the emotional challenges that inspired the paper’s best practices emphasizing not repeating negative behaviors, and Juniper notes tension in research between benefits for the community and for the researchers emphasizing understanding norms and values for studying open source communities.

[00:13:52] Richard mentions there are nine principles in the paper and asks about the principle regarding treating open source ecosystems as systems “in production.” Amanda highlights the importance of considering the real-world impact of research in open source and mentions an incident where a university was banned from the Linux kernel due to disruptive changes.

[00:16:33] Julie emphasizes the potential broader impact on industry systems when modifying open source systems and she raises the point that tampering with open source systems might inadvertently affect critical infrastructure. Amanda comments on the increasing cybersecurity concerns around open source.

[00:19:18] Richard brings up a real-world example of a university introducing bugs to the Linux kernel and points out the need for considering ethical implications beyond just production systems.

[00:20:59] Richard draws parallels between addressing these issues and addressing racism, and Juniper adds that the scientific process is ongoing and should evolve with technology and societal values.

[00:21:53] Julie describes the complexity of open source funding and compensation and points out the challenge in understanding motivations and expectations of open source participants.

[00:24:07] Amanda emphasizes the difficulty of summarizing each section, noting that each one could be a chapter or book and she expresses her concerns about not just individual equity but organizational equity.

[00:25:59] Juniper raises the issue of invisible labor in open source.

[00:26:39] Julie highlights the importance of recognizing that open source repository data might not capture all the activity and contributions made by community members.

[00:27:37] Amanda discusses the challenges and importance of capturing data, especially when it may put individuals at risk. Juniper stresses the importance of involving communities in the research process and gaining their consent, ensuring their dignity, security, and privacy.

[00:29:49] Julie discusses the complexities of identity within the open source community, highlighting that individuals can hold multiple identities in this space.

[00:31:10] Richard adds that the insight shared are not just for open source researchers but also for anyone involved in the open source ecosystem. He emphasizes the need to be aware of biases and the importance of understanding the data one works with.

[00:32:22] Richard prompts a summary of the main points in the paper, which are read by our guests.

[00:34:48] Find out where you can learn more about our guests and their work online.

Quotes

[00:20:08] “Production as the end line for ethical values leads to a lot of really thorny edge cases that are going to ultimately hurt the communities of people who aren’t working on production ready systems.”

[00:21:20] “Just as open source is always in production, so is the scientific process.”

[00:23:24] “Even having the privilege of time to dedicate to open source is not available to all.”

[00:24:26] “It’s just not individual equity but organizational equity.”

[00:25:47] “We can’t ignore the very large industry that is open source that has all that money moving around and where it’s going is a question we should all be asking.”

[00:26:00] “There’s a lot of invisible labor in open source.”

[00:28:32] “Leaving out communities from the scientific process of the research process leaves open these vulnerabilities without giving them a voice to what kind of research is being done about them without their consent.”

[00:29:17] “What we are starting to consider acceptable surveillance in public is really being challenged.”

[00:29:33] “It’s really important for us to make sure that we’re maintaining people’s dignity, security, and privacy while we’re doing this kind of research.”

Spotlight

[00:35:45] Richard's spotlight is The Long Trail that he’s going to hike.
[00:36:17] Amanda’s spotlight is contributor-experience.org and the PyPI subpoena transparency report.
[00:37:20] Julie’s spotlight is the book, Data Feminism.
[00:38:09] Juniper’s spotlight is a new tool called, XGI.