AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
CDC Has Changed Data Capture
What we're seeing is a hybridization of batch and streaming. Some form of batch processing allows you to do more complex analysis because you can correlate data that would not naturally be within the same timeline though. This notion of hybridizing batch and streaming has actually started to get rolled up into the concept of data flows. For example, how we've implemented inside of a send is using high frequency micro-batch. We grab small chunks of data and move them through as efficiently as we can based off the availability of the data itself.