Tokio runtimes: work stealing vs no‑steal

episode 15 — Pingora with Edward and Noah from Cloudflare.

In Episode 15 of netstack.fm, Glen sits down with Edward and Noah from Cloudflare to unpack the design of Pingora, the Rust based proxy framework that now powers Cloudflare’s origin facing traffic. The discussion covers why Cloudflare moved away from NGINX, how Pingora differs from Oxy, and what it takes to operate a high performance global proxy at massive scale. Listeners will learn about connection reuse strategies, dynamic traffic handling, gRPC and protocol translation, custom HTTP implementations, TLS backend choices, and the practical trade offs of Rust, Tokio, and work stealing in real production systems. It is an episode full of deep technical insights into building and operating modern networking infrastructure.

Learn more:

• https://blog.cloudflare.com/how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet/ — Pingora's introduction blog
• https://blog.cloudflare.com/pingora-open-source/ — Pingora launch blog post (Cloudflare)
• https://github.com/vicanso/pingap — pingap, built on top of pingora
• https://blog.cloudflare.com/road-to-grpc/ — Cloudflare gRPC and gRPC Web article
• https://github.com/hyperium/h2 — Hyperium h2 crate
• https://github.com/cloudflare/quiche — Quiche (QUIC and HTTP3 implementation)
• https://blog.cloudflare.com/technical-breakdown-http2-rapid-reset-ddos-attack/ — Cloudflare Rapid Reset HTTP2 attack analysis
• https://csrc.nist.gov/pubs/fips/140-3/final — NIST FIPS 140-3 publication
• https://blog.cloudflare.com/introducing-oxy/ — Introducing Oxy (Cloudflare)
• https://blog.cloudflare.com/oxy-extensibility/ — Oxy's extensions via generics
• https://blog.cloudflare.com/20-percent-internet-upgrade/ — Oxidizing other CDN components for performance
• https://github.com/cloudflare/foundations — Cloudflare's foundations crate

Rama

If you like this podcast you might also like our modular network framework in Rust: https://ramaproxy.org

Chapters

• 00:00 Intro
• 00:37 A bit of background on the episode and our guests
• 03:18 The Evolution of Proxy Frameworks: Oxy vs. Pingora
• 14:59 The Philosophy Behind Pingora's Design
• 20:53 Understanding Pingora's Bare Bones Structure
• 27:50 Metrics and Observability in Pingora
• 39:19 Caching Strategies and Backend Structures
• 42:56 Usage of OnceCell
• 45:39 TLS Implementations and Their Importance
• 50:51 Dynamic Traffic Management and gRPC Support
• 01:02:10 Optimizing Connection Reuse with Pingora
• 01:07:10 The Importance of Layer 7 Processing
• 01:11:16 The Shift from Static to Dynamic Web Traffic
• 01:18:48 Performance Improvements with Rust and Tokio
• 01:26:00 Memory Management and Allocation Strategies
• 01:37:59 Outro

Netstack.FM

• More information: https://netstack.fm/#episode-15
• Join our Discord: https://discord.gg/29EetaSYCD
• Reach out to us: hello@netstack.fm

Music for this episode was composed by Dj Mailbox. Listen to his music at https://on.soundcloud.com/4MRyPSNj8FZoVGpytj