The Pain of Migrating From Open Source to Open Source

eBay is a high scale end user of open source observability. Collecting telemetry from millions of endpoints, and running thousands of queries per second, presents serious scaling challenges. eBay has chosen to use an open source stack to meet those challenges. On this episode I hosted Vijay Samuel, Observability Architect at eBay, to hear about the challenges eBay faced in monitoring large Kubernetes installations. We discussed why eBay chose the open source stack, and the strategic decision behind eBay's migration from Elastic Beats to OpenTelemetry, and the remarkable experiences they had while deploying massively large scale telemetry installations using OpenTelemetry and Prometheus. We also discussed a fascinating new proposal for a query language standard for observability, which eBay spearheads together with Netflix under the auspices of the CNCF’s TAG Observability (the CNCF’s technical advisory group for observability).

The episode was live-streamed on 8 June 2023 and the video is available at https://www.youtube.com/watch?v=-UsU3nRglhA

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.

https://www.twitch.tv/openobservability

https://www.youtube.com/@openobservabilitytalks  

Show Notes:

3rd anniversary

eBay monitoring solution 

planet scale at eBay in numbers

distributed tracing at eBay

migrating from Elastic Beats to OpenTelemetry

why eBay chose open source 

open-sourcing eBay’s metrics store platform?

scaling Prometheus

ramping up tracing, from backend to frontend and mobile

running OpenTelemetry Collector at scale

Query Language Standardization for Observability under the CNCF

Resources:

New CNCF working group for Observability Query Language Standardization: https://github.com/cncf/tag-observability/blob/main/working-groups/query-standardization.md Why and How eBay Pivoted to OpenTelemetry: https://tech.ebayinc.com/engineering/why-and-how-ebay-pivoted-to-opentelemetry/

Socials:

• Twitter: https://twitter.com/OpenObserv

• Twitch: https://www.twitch.tv/openobservability

• YouTube: https://www.youtube.com/@openobservabilitytalks

Dotan Horovits

============

Twitter: @horovits

LinkedIn: in/horovits

Mastodon: @horovits@fosstodon

Vijay Samuel

==========

Twitter: @vjsamuel_

LinkedIn: vjsamuel