DataNation - Podcast for Data Engineers, Analysts and Scientists

51 – Open Data Standards (Apache Iceberg, Apache Parquet, Apache Arrow, Apache Ibis, Apach Substrait)

7 snips
Mar 18, 2024
Explore the benefits of open data standards like Apache Arrow and Apache Iceberg in the data space, optimizing data transfer efficiency with Apache Arrow Flight and ADBC, enhancing scan planning in data catalogs with Apache Iceberg spec and Apache Ibis, standardizing data frameworks and SQL query processing with Apache Substrate, and the value of standardized open data formats and systems for innovation and efficiency.
Ask episode
Chapters
Transcript
Episode notes
1
Introduction
00:00 • 2min
2
Optimizing Data Transfer with Apache Arrow Flight and ADBC
02:04 • 6min
3
Optimizing Scan Planning in Data Catalogs
08:31 • 5min
4
Advantages of Standardization in Data Frameworks and SQL Query Processing
13:43 • 2min
5
The Value of Standardized Open Data Formats and Systems
15:52 • 2min