Unevenly Distributed #1: How Open Lakehouses Are Disrupting the Data Ecosystem ft. Brad Heller (Co-founder & CTO, Tower)
19 snips
Aug 21, 2025 Brad Heller, Co-founder and CTO of Tower, shares his expertise in serverless Python infrastructure and the evolution of data management. The discussion centers on why teams are rethinking their data platforms, exploring how open lakehouses are challenging the traditional data ecosystem. Heller delves into the benefits of open formats, the complexities of multi-cloud management, and the vital role of metadata services. He highlights the future of data engineering and the skills needed to thrive in this changing landscape.
AI Snips
Chapters
Transcript
Episode notes
Hadoop Sparked The Data Lake Era
- Hadoop introduced distributed storage and MapReduce to handle exploding web-scale data volumes.
- That shift enabled storing massive logs and led to the emergence of data lakes for later analysis.
Data Lakes Retained Too Much
- The data lake concept let teams dump data and define use later, leading to massive retained volumes.
- That durability created a long-term storage burden where much data remains unused.
Customer With Petabytes Of Forgotten Data
- Brad recounts a customer with thousands of petabytes of historical transaction data sitting unused.
- They are now struggling in 2025 to decide what to keep and what to delete.