Change Data Capture and Stream Processing in the Cloud - Gunnar Morling from Decodable
Apr 26, 2024
auto_awesome
Gunnar Morling from Decodable shares insights on change data capture using Debezium and stream processing with Apache Flink. Topics cover real-time data analytics, seamless integration with data systems like Postgres and MySQL, and the importance of unified change event formats. Gunnar also discusses Java performance tests, Apache Flink for processing real-time data streams, and trends in database evolution and microservices.
Change data capture tools like Debezium extract modifications from databases for real-time analytics.
Apache Flink enables stream processing of bounded and unbounded event streams with varied APIs.
Deep dives
Gunnar's Background and Role at Decodable
Gunnar, a software engineer at Decodable, discusses his role at the startup focused on data streaming. He elaborates on his experience at Red Hat where he worked on various projects under the Hibernate umbrella, including bean validation and Debezium, a change data capture tool. Gunnar mentions his involvement in outreach activities such as blog posts, podcasts, and conferences.
Decodable's Data Streaming Service
Decodable offers a SaaS platform for real-time stream processing connecting to databases and streaming platforms like Kafka and Apache Pulsar. They facilitate data movement across different systems in real-time with low latency, enabling tasks such as filtering, transformation, and real-time analytics. Decodable leverages Apache Flink as a stream processing engine and Debezium for change data capture, catering to use cases like fraud detection and customer insights.
Apache Flink and Debezium Overview
Gunnnar explains Apache Flink as an open-source project for real-time data processing, handling bounded and unbounded event streams with imperative, relational, and SQL-based APIs. He also touches on Debezium, a change data capture tool that captures and propagates database change events to consumers, aiding replication, microservices, and real-time data analysis. Gunnar highlights the tool's versatility, supporting various databases and promoting a standardized event format for unified processing.
Gunnar Morling, from Decodable, a company simplifying the use of Apache Flink, talks about how to extract modifications from a database using change data capture tools, such as Debezium, and how to use Apache Flink or Decodable to run your real-time analytics or queries in a stream processing fashion.
If you have questions to Gunnar, you can reach him here: