AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Screaming in the Cloud
S3 and the Evolution of Storage with Andy Warfield
Feb 4, 2025
Andy Warfield, Vice President and Distinguished Engineer at Amazon Web Services, discusses the groundbreaking evolution of storage technology. He highlights how S3 has transformed from archival storage to a powerhouse for modern AI and analytics. The conversation covers innovative features like S3 Tables and the Common Runtime (CRT), along with challenges such as namespace structuring. Andy shares insights from his journey in tech, revealing how S3's scalability now employs millions of hard disks, while reflecting on the frustrations of legacy applications.
36:39
Episode guests
AI Summary
AI Chapters
Episode notes
Podcast summary created with Snipd AI
Quick takeaways
- The evolution of S3 reflects a shift from basic storage to advanced analytics capabilities, incorporating formats like Iceberg for enhanced data management.
- Amazon's adaptability to customer feedback has driven innovations in S3's functionality and API design, ensuring they meet evolving use cases effectively.
Deep dives
The Evolution of Data Lakes and Performance Optimization
Data lakes built on S3 have seen significant interest from customers, particularly regarding Parquet performance optimizations. Over the years, a transition occurred as users shifted from Parquet to more sophisticated formats like Iceberg and Open Table formats, indicating a demand for better data management solutions. As these formats gained traction, it became clear that customers were seeking enhancements in their data analytics and storage capabilities, prompting developments like the S3 Tables product. This evolution reflects the changing landscape of how organizations are leveraging cloud storage for complex analytics workloads.
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode