Data Engineering Podcast The Benefits And Challenges Of Building A Data Trust
Feb 3, 2020
56:53
Summary
Every business collects data in some fashion, but sometimes the true value of the collected information only comes when it is combined with other data sources. Data trusts are a legal framework for allowing businesses to collaboratively pool their data. This allows the members of the trust to increase the value of their individual repositories and gain new insights which would otherwise require substantial effort in duplicating the data owned by their peers. In this episode Tom Plagge and Greg Mundy explain how the BrightHive platform serves to establish and maintain data trusts, the technical and organizational challenges they face, and the outcomes that they have witnessed. If you are curious about data sharing strategies or data collaboratives, then listen now to learn more!
Announcements
- Hello and welcome to the Data Engineering Podcast, the show about modern data management
- When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With 200Gbit private networking, scalable shared block storage, and a 40Gbit public network, you’ve got everything you need to run a fast, reliable, and bullet-proof data platform. If you need global distribution, they’ve got that covered too with world-wide datacenters including new ones in Toronto and Mumbai. And for your machine learning workloads, they just announced dedicated CPU instances. Go to dataengineeringpodcast.com/linode today to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
- You listen to this show to learn and stay up to date with what’s happening in databases, streaming platforms, big data, and everything else you need to know about modern data management. For even more opportunities to meet, listen, and learn from your peers you don’t want to miss out on this year’s conference season. We have partnered with organizations such as O’Reilly Media, Corinium Global Intelligence, ODSC, and Data Council. Upcoming events include the Software Architecture Conference in NYC, Strata Data in San Jose, and PyCon US in Pittsburgh. Go to dataengineeringpodcast.com/conferences to learn more about these and other events, and take advantage of our partner discounts to save money when you register today.
- Your host is Tobias Macey and today I’m interviewing Tom Plagge and Gregory Mundy about BrightHive, a platform for building data trusts
Interview
- Introduction
- How did you get involved in the area of data management?
- Can you start by describing what a data trust is?
- Why might an organization want to build one?
- What is BrightHive and what is its origin story?
- Beyond having a storage location with access controls, what are the components of a data trust that are necessary for them to be viable?
- What are some of the challenges that are common in establishing an agreement among organizations who are participating in a data trust?
- What are the responsibilities of each of the participants in a data trust?
- For an individual or organization who wants to participate in an existing trust, what is involved in gaining access?
- How does BrightHive support the process of building a data trust?
- How is ownership of derivative data sets/data products and associated intellectual property handled in the context of a trust?
- How is the technical architecture of BrightHive implemented and how has it evolved since it first started?
- What are some of the ways that you approach the challenge of data privacy in these sharing agreements?
- What are some legal and technical guards that you implement to encourage ethical uses of the data contained in a trust?
- What is the motivation for releasing the technical elements of BrightHive as open source?
- What are some of the most interesting, innovative, or inspirational ways that you have seen BrightHive used?
- Being a shared platform for empowering other organizations to collaborate I imagine there is a strong focus on long-term sustainability. How are you approaching that problem and what is the business model for BrightHive?
- What have you found to be the most interesting/unexpected/challenging aspects of building and growing the technical and business infrastructure of BrightHive?
- What do you have planned for the future of BrightHive?
Contact Info
- Tom
- Gregory
- gregmundy on GitHub
- @graygoree on Twitter
Parting Question
- From your perspective, what is the biggest gap in the tooling or technology for data management today?
Closing Announcements
- Thank you for listening! Don’t forget to check out our other show, Podcast.__init__ to learn about the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com) with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
- Join the community in the new Zulip chat workspace at dataengineeringpodcast.com/chat
Links
- BrightHive
- Data Science For Social Good
- Workforce Data Initiative
- NASA
- NOAA
- Data Trust
- Data Collaborative
- Public Benefit Corporation
- Terraform
- Airflow
- Dagster
- Secure Multi-Party Computation
- Public Key Encryption
- AWS Macie
- Blockchain
- Smart Contracts
The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA