2min snip

Monday Morning Data Chat cover image

#137 - Why Apache Iceberg Won the Table Format War, Data Mesh, and More w/ Brian Olsen

Monday Morning Data Chat

NOTE

Building compatibility for metadata layer and support for iceberg files

Developing compatibility with iceberg involves supporting the metadata layer and building Avro files that act as a tree structure, where the root sits in a database or catalog with nodes scattered across S3. These nodes are pointers to other nodes on the tree leading to the data node for query processing. Support for iceberg can range from using it as an external table or external engine, to building metadata on top of proprietary storage, to storing data in an interchange format, and finally storing it at the file level in a non-proprietary open file format like parquet.

00:00

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode