Navigating the Challenges of Live Streaming Scalability

Supported by Our Partners

• WorkOS — The modern identity platform for B2B SaaS

• CodeRabbit — Cut code review time and bugs in half

• Augment Code — AI coding assistant that pro engineering teams love

—

How do you architect a live streaming system to deal with more load than it’s ever been done before? Today, we hear from an architect of such a system: Ashutosh Agrawal, formerly Chief Architect of JioCinema (and currently Staff Software Engineer at Google DeepMind.)

We take a deep dive into video streaming architecture, tackling the complexities of live streaming at scale (at tens of millions of parallel streams) and the challenges engineers face in delivering seamless experiences. We talk about the following topics: 

• How large-scale live streaming architectures are designed

• Tradeoffs in optimizing performance

• Early warning signs of streaming failures and how to detect them

• Why capacity planning for streaming is SO difficult

• The technical hurdles of streaming in APAC regions

• Why Ashutosh hates APMs (Application Performance Management systems)

• Ashutosh’s advice for those looking to improve their systems design expertise

• And much more!

—

Timestamps

(00:00) Intro

(01:28) The world record-breaking live stream and how support works with live events

(05:57) An overview of streaming architecture

(21:48) The differences between internet streaming and traditional television.l

(22:26) How adaptive bitrate streaming works

(25:30) How throttling works on the mobile tower side 

(27:46) Leading indicators of streaming problems and the data visualization needed

(31:03) How metrics are set 

(33:38) Best practices for capacity planning 

(35:50) Which resources are planned for in capacity planning 

(37:10) How streaming services plan for future live events with vendors

(41:01) APAC specific challenges

(44:48) Horizontal scaling vs. vertical scaling 

(46:10) Why auto-scaling doesn’t work

(47:30) Concurrency: the golden metric to scale against

(48:17) User journeys that cause problems 

(49:59) Recommendations for learning more about video streaming 

(51:11) How Ashutosh learned on the job

(55:21) Advice for engineers who would like to get better at systems

(1:00:10) Rapid fire round

—

The Pragmatic Engineer deepdives relevant for this episode:

• Software architect archetypes https://newsletter.pragmaticengineer.com/p/software-architect-archetypes 

• Engineering leadership skill set overlaps https://newsletter.pragmaticengineer.com/p/engineering-leadership-skillset-overlaps 

• Software architecture with Grady Booch https://newsletter.pragmaticengineer.com/p/software-architecture-with-grady-booch

—

See the transcript and other references from the episode at ⁠⁠https://newsletter.pragmaticengineer.com/podcast⁠⁠

—

Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email podcast@pragmaticengineer.com.