DataNation - Podcast for Data Engineers, Analysts and Scientists 63 – Reinvent, AWS S3 Table Buckets and Apache Iceberg
4 snips
Dec 6, 2024 The guest shares vibrant highlights from the AWS re:Invent 2024 event. Key announcements feature S3 Table Buckets and their implications for data catalogs. Innovations in AWS Glue allow for multiple catalogs, enhancing data management flexibility. The conversation also touches on Apache Iceberg's growth, emphasizing Dremio's role in streamlining migration challenges. This creates a smoother experience for data engineers and users alike.
AI Snips
Chapters
Transcript
Episode notes
First re:Invent
- Alex Merced attended his first AWS re:Invent in Las Vegas.
- The expo hall was massive, filled with vendors vying for attention.
S3 Table Buckets
- AWS announced S3 Table Buckets, a new S3 bucket type for Apache Iceberg Tables.
- Concerns arose about moving away from catalogs, but AWS clarified their approach.
AWS Glue Changes
- AWS Glue is changing; instead of one catalog per account, multiple catalogs/sub-catalogs will be supported.
- S3 table buckets act as sub-catalogs within a main catalog, maintaining interoperability.