Day Two DevOps cover image

Day Two DevOps

D2DO252: (Re)Building Cloudflare’s Millions-of-Logs-Per-Second Logging Pipeline

Oct 2, 2024
Colin Douch, Observability Tech Lead at Cloudflare, and Jayson Cena, SRE at Cloudflare, dive into the complexities of migrating from Syslog-NG to OpenTelemetry. They discuss the motivations for this shift, such as scalability and memory safety, while tackling challenges like maintaining uninterrupted customer traffic. The duo also highlights the importance of redundancy in logging systems and shares insights on logging protocols, illustrating the balance between resource usage and operational speed in a high-performance environment.
37:53

Podcast summary created with Snipd AI

Quick takeaways

  • Cloudflare's migration to OpenTelemetry significantly enhances its logging capabilities by improving scalability, performance, and maintainability for handling millions of logs per second.
  • The successful deployment of OpenTelemetry required meticulous planning to ensure uninterrupted customer traffic, showcasing the importance of operational efficiency during major transitions.

Deep dives

Importance of Managing Unmanaged Devices and Apps

Companies face significant challenges in securing data when employees use unmanaged devices and non-approved applications. Traditional identity and access management (IAM) and mobile device management (MDM) solutions often fall short in addressing these security gaps. It is essential to implement strategies that extend beyond conventional methods to safeguard sensitive information effectively. Organizations need to prioritize solutions that cater to the complexities of modern work environments.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode