#164 - Jean-Georges Perrin - Data Mesh, Data Contracts, Modern Data Engineering Standards, Bitol, and More
Feb 26, 2024
auto_awesome
In this podcast, Jean-Georges Perrin discusses data mesh, data contracts, modern data engineering standards, and the open standard project Bitol. They also explore data architecture and the shift towards practices over technology. The conversation extends to PayPal's adoption of a data mesh approach, challenges faced, and successful data product launches within set constraints.
Data contracts foster efficient data engineering environments through trust and communication.
Embracing new technologies and methodologies is essential for evolving data engineering standards.
Transitioning towards data mesh architecture decentralizes data platforms and enhances user-driven product creation.
Deep dives
Overview of the Podcast Episode
The podcast episode featured a discussion between the two hosts, Joe and Sean George, covering a wide range of topics from personal anecdotes to insights into the tech industry. Sean George, a software engineer turned data engineer, shared his journey from entrepreneurship to consulting and his new ventures. The conversation touched on the startup scenes in France and the US and the challenges of access to capital in entrepreneurial endeavors.
Evolution of Modern Data Engineering Practices
Sean George delved into modern data engineering practices, emphasizing the need for a shift in mindset from traditional data handling approaches. He highlighted the importance of agility, product thinking, and the role of data contracts in fostering a more efficient and innovative data engineering environment. The discussion explored the significance of embracing new technologies and methodologies in evolving data engineering standards.
Importance of Data Contracts
The conversation delved into the concept of data contracts as a crucial component in modern data engineering. Sean George described data contracts as computational brochures that establish trust and communication between data producers and consumers. Emphasizing the role of data quality of service, he underscored the significance of standardizing communication to enhance data interoperability and innovation.
Transition Towards Data Mesh Architecture
Sean George shared insights into the transition towards data mesh architecture, detailing its impact on data platform decentralization and user-driven data product creation. The discussion encompassed the sociotechnical aspects of data mesh, focusing on architectural implementation, team topologies, and organizational strategies for managing decentralized data products.
Upcoming Book Project on Implementing Data Mesh
Sean George provided a sneak peek into his upcoming book project with Eric Broder on implementing data mesh. Scheduled for release in October or November, the book aims to offer a comprehensive guide to data mesh architecture, emphasizing technical implementation, architectural considerations, and organizational management strategies. Sean George highlighted the book's focus on a climate change use case, architectural insights, and team dynamics within a data mesh framework.
Jean-Georges Perrin joins the show to chat about data mesh, data contracts, modern data engineering standards, Bitol (is open standard project with the Linux Foundation), data architecture, and much more. This is a wide-reaching discussion. Enjoy!
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode