In the final episode of our 3-part series on the basics of data streaming, we take a deep dive into data integration—covering everything from data governance to data quality.
Our guests, Mike Agnich, General Manager of Data Streaming Platform, and David Araujo, Director of Product Management at Confluent, explain why connectors are must-haves for integrating systems.
You’ll learn:
- Why real-time ETL out performs the old-school approach
- How shifting left with governance saves time and pain later
- The overlooked role of schemas in data quality
- And more…
About the Guests:
Mike Agnich is the General Manager and VP of Product for Confluent's Data Streaming Platform (DSP). Mike manages a product portfolio that includes stream processing, connectors and integrations, governance, partnerships, and developer tooling. Over the last six years at Confluent, Mike has held various product leadership roles spanning Apache Kafka®, Confluent Cloud, and Confluent Platform. Working closely with customers, partners, and R&D to drive adoption and execution of Confluent products. Prior to his work at Confluent, Mike was the founder and CEO of Terrain Data (acquired by Confluent in 2018).
David Araujo is a Director of Product Management at Confluent, focusing on data governance with products such as Schema Registry, Data Catalog, and Data Lineage. David previously held positions at companies like Amobee, Turn, WeDo Technologies Australia, and Saphety, where David worked on various aspects of data management, analytics, and infrastructure. With a background in Computer Science from the University of Évora, David has a strong foundation of technical expertise and leadership roles in the tech industry.
Guest Highlights:
"If a ton of raw data shows up on your doorstep, it's like shipping an unlabeled CSV into a finance organization and telling them to build their annual forecast. By shifting that cleaning and structure into streaming, we remove a massive amount of toil for our organizations… Instead of punting the problem down to our analytics friends, we can solve it because we're the ones that created the data." - Mike Agnich
"We've had data contracts in Kafka long before it became a buzzword—we called them schemas… But more recently, we've evolved this concept beyond just schemas. In streaming, a data contract is an agreement between producers and consumers on both the structure (schema) and the semantics of data in motion. It serves as a governance artifact, ensuring consistency, reliability, and quality while providing a single source of truth for understanding streaming data." - David Araujo
Links & Resources:
Episode Timestamps:
*(02:00) - Mike and David’s Journey in Data Streaming
*(13:55) - Data Streaming 101: Data Integration
*(40:06) - The Playbook: Tools & Tactics for Data Integration
*(53:25) - Voices from the World of Data Streaming
*(59:33) - Quick Bytes
*(1:05:20) - Joseph’s Top 3 Takeaways
Our Sponsor:
Your data shouldn’t be a problem to manage. It should be your superpower. The Confluent Data Streaming Platform transforms organizations with trustworthy, real-time data that seamlessly spans your entire environment and powers innovation across every use case. Create smarter, deploy faster, and maximize efficiency with a true Data Streaming Platform from the pioneers in data streaming. Learn more at confluent.io
Life Is But A Stream