Mastering Data at Scale: Uber's Evolving Architecture

15min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

This chapter explores Uber's journey in navigating their tech stack for analytics, highlighting the vital role of HBase and effective data management practices. It discusses the significant integration of technologies like Spark, Airflow, and Kafka, as well as the introduction of Ledger Store for tracking immutable datasets. Additionally, the conversation addresses the complexities of scaling, data migration, and maintaining consistency across vast amounts of data, emphasizing the need for tailored solutions in an ever-growing environment.

Uber handles billions of trips and deliveries, and tens of billions of financial transactions across drivers, couriers, users, and merchants every quarter.

LedgerStore is an immutable storage solution at Uber that provides verifiable data completeness and correctness guarantees to ensure data integrity for its transactions.

Kaushik Devarajaiah is the Tech Lead for LedgerStore at Uber. He joins the show to talk about scaling Uber’s data and storage infrastructure.

Sean’s been an academic, startup founder, and Googler. He has published works covering a wide range of topics from information visualization to quantum computing. Currently, Sean is Head of Marketing and Developer Relations at Skyflow and host of the podcast Partially Redacted, a podcast about privacy and security engineering. You can connect with Sean on Twitter @seanfalconer.