Tech on the Rocks

Amey Chaugule and Matt Green, co-founders of Denormalized, share their extensive engineering backgrounds from top tech firms. They discuss the creation of an embedded stream processing engine designed to simplify real-time data workloads. The duo tackles challenges in existing systems like Spark and Kafka, emphasizing developer experience and state management. They also compare DuckDB and SQLite in the context of streaming data, highlighting the future of user-friendly data tools and the importance of fault tolerance in modern applications.

Nikhil Simha and Varant Zanoyan, both seasoned engineers with rich backgrounds in data systems and ML infrastructure, discuss the intricate balance of structured and unstructured data in AI. They delve into the challenges of merging real-time data with machine learning, emphasizing the importance of user-friendly APIs. The conversation touches on failures in data transformation and effective strategies for startups to engage users. They also introduce Cronon, an open-source platform, highlighting its potential to improve data orchestration and user experience.

Chris Riccomini, an expert in stream processing and LSMs, dives into the evolution of streaming systems, highlighting the challenges developers face. He critiques SQL's limitations in this space and emphasizes the need for better API designs. The discussion also touches on the impact of Rust on usability and efficiency, particularly in embedded libraries. Chris shares insights about his exciting project involving log-structured merge trees on object storage, and the future of data systems with a focus on composable databases and the importance of metadata in AI.