The Analytics Engineering Podcast

How Amazon S3 works (w/ Andy Warfield)

46 snips
Jul 20, 2025
Andy Warfield, VP and Senior Principal Engineer at AWS, sheds light on the inner workings of Amazon S3, a vital player in cloud data management. He discusses the evolution of S3, its early misconceptions, and how customer feedback has guided its technology roadmap. The conversation highlights the significance of S3 table buckets in AI and ML applications and the integration with formats like Iceberg. Finally, they explore the future of storage solutions and how S3 continues to power advancements in data analytics.
Ask episode
AI Snips
Chapters
Transcript
Episode notes
ANECDOTE

Andy Warfield's Early Computing Journey

  • Andy Warfield's love for low-level computing began with cobbling together garage sale computer parts in high school in Ottawa.
  • His early entrepreneurial efforts included reselling old government office equipment, learning a lot about hardware and software in the process.
INSIGHT

Why Virtualization Took Off

  • Virtualization grew in the 2000s to improve server utilization and isolate enterprise applications.
  • Intel's eventual support with VT-x enabled efficient virtualization on x86 CPUs despite initial architectural unsuitability.
INSIGHT

S3's Initial Design Philosophy

  • S3 started as an internet file system akin to a secure storage locker with slower, less responsive access.
  • Early use cases included backups and media origins supported by CDNs to mitigate latency challenges.
Get the Snipd Podcast app to discover more snips from this episode
Get the app