Monday Morning Data Chat cover image

#137 - Why Apache Iceberg Won the Table Format War, Data Mesh, and More w/ Brian Olsen

Monday Morning Data Chat

00:00

Building compatibility for metadata layer and support for iceberg files

Developing compatibility with iceberg involves supporting the metadata layer and building Avro files that act as a tree structure, where the root sits in a database or catalog with nodes scattered across S3. These nodes are pointers to other nodes on the tree leading to the data node for query processing. Support for iceberg can range from using it as an external table or external engine, to building metadata on top of proprietary storage, to storing data in an interchange format, and finally storing it at the file level in a non-proprietary open file format like parquet.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app