Evolution of Distributed Systems and Database Scaling Challenges

6min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

This chapter explores the origins of distributed systems like Spanner, custom solutions required for scaling, and the challenges faced in horizontal sharding for database scaling. It also delves into the importance of selecting a simple feature set for initial deployment and the complexity of cross-shard transactions with a focus on maintaining data integrity during migrations.

Sammy Steele is a Senior Staff Engineer at Figma, and the tech lead for their databases team. She previously worked at Dropbox, where she built out their petabyte-scale metadata storage and search systems.

Sammy recently published a blog called “How Figma’s databases team lived to tell the scale”. The blog went viral and made it to the top of Hacker News. We invited Sammy on the podcast to learn more, and she is our guest today.

Check out Figma Config at https://config.figma.com/.

Sean’s been an academic, startup founder, and Googler. He has published works covering a wide range of topics from information visualization to quantum computing. Currently, Sean is Head of Marketing and Developer Relations at Skyflow and host of the podcast Partially Redacted, a podcast about privacy and security engineering. You can connect with Sean on Twitter @seanfalconer .