The Logan Bartlett Show cover image

EP 89: Jay Kreps (CEO, Confluent) on Confluent’s Resilient Rise to Software Behemoth

The Logan Bartlett Show

NOTE

Real-time Processing of Data Streams

The main insight is that traditional data processing focused on storing data and batch processing, where data was processed periodically. However, with the rise of complex software systems consisting of multiple pieces of software with their own data, the focus shifted to real-time processing of data streams to react to events as they happen. This real-time processing of data streams was seen as a natural generalization of database ideas, and while it was previously considered research-oriented, the insights led to the development of a project internally at LinkedIn and its subsequent open-sourcing to the rest of the world.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner