OpenObservability Talks Shopify’s Journey to Planet-Scale Observability - OpenObservability Talks S5E09
Feb 27, 2025
Elijah McPherson, an engineering leader at Shopify with expertise in observability, discusses the company's ambitious journey to create an in-house observability stack called Observe. He dives into the challenges of vendor lock-in and the strategic pivot to open-source solutions. Elijah emphasizes the critical role of developer experience and user feedback in building tools that meet engineers' needs. The conversation highlights innovative features like the Investigate API and the importance of integrating business metrics for enhanced performance insights.
AI Snips
Chapters
Transcript
Episode notes
High Observability Costs
- Shopify used three vendors for observability: logs, tracing, and metrics.
- The costs were astronomical and rising disproportionately to Shopify’s growth.
Tailoring for Needs
- Tailoring observability to organizational needs and empowering teams to own reliability are key drivers for in-house solutions.
- Generic third-party tools may not fully align with unique workflows and internal processes.
Team Composition
- Build diverse teams with expertise in backend, frontend, UI/UX, operations, and product management when tackling large-scale internal platforms.
- Consider the operational aspects of running the platform at scale and providing internal support.