#171 - David Yaffe & John Kutay - The State of Streaming and Change Data Capture
Apr 22, 2024
auto_awesome
David Yaffe from Estuary and John Kutay from Striim discuss the state of streaming and Change Data Capture in 2024. They cover topics such as advancements in CDC technology, managed solutions for streaming data, challenges in data engineering, and the impact of accessible streaming on analytics and application development.
Real-time data processing tools play key roles in facilitating analytics.
Increasing accessibility and adoption of streaming technologies.
Shift towards real-time analytics driven by change data capture tools.
Deep dives
Evolution of Change Data Capture and Streaming Technologies
The podcast delves into the evolution and significance of change data capture and streaming technologies. It discusses how these technologies have progressed over time, outlining the key roles they play in facilitating real-time data processing and analytics.
State of Streaming and Change Data Capture in Mid-2024
The episode explores the current state of streaming and change data capture technologies as of mid-2024. It highlights the key advancements and trends in the field, emphasizing the increasing accessibility and adoption of real-time data processing tools.
Transition Towards Real-Time Analytics
The conversation points towards a shift towards real-time analytics driven by the availability and ease of use of change data capture and streaming tools. It underscores the potential of these technologies in enabling organizations to generate value through revenue-generating AI-driven products and operational efficiencies.
Challenges and Anti-Patterns in Data Streaming
The podcast addresses the challenges and common anti-patterns encountered in data streaming, such as inefficient ORM tool usage, long-running transactions, and inadequate communication between technical teams. It emphasizes the importance of addressing technical debt and optimizing data processes.
Future Trends and Predictions in Data Integration
In closing, the speakers offer insights into the future trends and predictions for data integration, including a move towards democratizing streaming technologies and integrating analytics with application development. They foresee a shift towards event-driven data processing and the convergence of operational and analytical approaches in data engineering.
David Yaffe (Estuary) and John Kutay (Striim) join the show to chat about the state of streaming and change data capture (CDC) in 2024 and beyond. There is a lot to cover and learn in this show.
Estuary: https://estuary.dev/
Striim: https://www.striim.com/
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode