The Analytics Engineering Podcast How Amazon S3 works (w/ Andy Warfield)
49 snips
Jul 20, 2025 Andy Warfield, VP and Senior Principal Engineer at AWS, sheds light on the inner workings of Amazon S3, a vital player in cloud data management. He discusses the evolution of S3, its early misconceptions, and how customer feedback has guided its technology roadmap. The conversation highlights the significance of S3 table buckets in AI and ML applications and the integration with formats like Iceberg. Finally, they explore the future of storage solutions and how S3 continues to power advancements in data analytics.
AI Snips
Chapters
Transcript
Episode notes
Andy Warfield's Early Computing Journey
- Andy Warfield's love for low-level computing began with cobbling together garage sale computer parts in high school in Ottawa.
- His early entrepreneurial efforts included reselling old government office equipment, learning a lot about hardware and software in the process.
Why Virtualization Took Off
- Virtualization grew in the 2000s to improve server utilization and isolate enterprise applications.
- Intel's eventual support with VT-x enabled efficient virtualization on x86 CPUs despite initial architectural unsuitability.
S3's Initial Design Philosophy
- S3 started as an internet file system akin to a secure storage locker with slower, less responsive access.
- Early use cases included backups and media origins supported by CDNs to mitigate latency challenges.