OpenObservability Talks

In this episode, we’ll talk with industry veteran and product manager Anurag Gupta who has been working in open source observability for over 4 years. We will go into depth on his background, and how he views the ecosystem of open source. Then we will dig into the Fluentd and Fluent Bit projects and discuss some of the amazing innovations coming from this project. Learn what’s next for logging, and how a consolidated data collection plane is being driven by the Fluentd project.

The CNCF has a rich suite to address monitoring Kubernetes and cloud-native workloads. First of which is Prometheus, which is widely adopted, with great out-of-the-box compatibility with Kubernetes. But under the CNCF you can also find OpenMetrics that offers standardization of the metrics format, Thanos and Cortex which offer long-term storage for Prometheus, and other complimentary solutions and integrations.    On this episode of OpenObservability Talks we’ll host “RichiH” Hartmann and discuss the different OSS projects, the synergy between them, and the future roadmap in building the community and making CNCF a leading offering.   Richard "RichiH" Hartmann is Director of Community at Grafana Labs, Prometheus team member, OpenMetrics founder, CNCF SIG Observability chair, and other things. He also organizes various conferences, including FOSDEM, DENOG, DebConf, and Chaos Communication Congress. In the past, he made mainframe databases work, ISP backbones run, and built a datacenter from scratch. The episode was live-streamed on 02 July 2021 and the video is available at https://youtube.com/live/j3nFFHSosnI Show Notes: OpenTelemetry accepted to CNCF incubation OpenTelemetry structure OpenTelemetry community adoption OpenMetrics and Open* confusion OpenMetrics and OpenTelemetry synergy OpenMetrics updates CNCF’s Observability TAG (Technical Advisory Group) How to sync between projects on CNCF Prometheus state and roadmap Prometheus conformance program Thanos and Cortex projects how the tech stack benefits humans Grafana, Loki and Tempo projects Resources: OpenTelemetry.io OpenTelemetry status page Guide to OpenTelemetry CNCF TAG Observability Open* Explainer by RichiH OpenMetrics

Current observability practice is largely based on manual instrumentation, which creates a barrier to entry for many wishing to implement observability in their environment. This is especially true in Kubernetes environments and microservices architecture. eBPF (extended Berkeley Packet Filter) is an exciting new technology for Linux kernel level instrumentation, which bears the promise of no-code instrumentation and easier observability into Kubernetes environments (alongside other benefits for networking and security). On this episode of OpenObservability Talks we’ll host Natalie Serrino, Principal Engineer at Pixie Labs, which was recently acquired by New Relic. We’ll talk about observability in Kubernetes environments, eBPF and its use cases for observability. We’ll also talk about Pixie, the Kubernetes-native in-cluster observability platform, and the exciting news of it being open sourced and contributed these days to CNCF under Apache 2.0 license. Natalie is a Principal Engineer and Tech Lead at New Relic. She works on the Pixie auto-telemetry observability platform, which was acquired and open sourced by New Relic. She focuses primarily on Pixie’s data layer, including its query language, compiler, and query execution engine. The episode was live-streamed on 20 June 2021 and the video is available at https://youtube.com/live/NYDBj5ctKaw Show Notes: challenges in k8s observability state of instrumentation automatic instrumentation eBPF overview eBPF vs. service mesh side cars Pixie project overview Pixie’s roadmap and integration plans with CNCF ecosystem Netflix engineering sharing use case of eBPF instrumenting with Istio opensearch RC1 released K8s unpredictable spend logs aren't enough, need tracing - recommended article Resources: http://www.brendangregg.com/ebpf.html https://blog.px.dev/ https://docs.px.dev/about-pixie/roadmap/ https://www.businesswire.com/news/home/20210504005480/en/New-Relic-Joins-Cloud-Native-Computing-Foundation-Governing-Board-and-is-in-the-Process-of-Contributing-Pixie-Open-Source-for-Kubernetes-Native-Observability https://netflixtechblog.com/how-netflix-uses-ebpf-flow-logs-at-scale-for-network-insight-e3ea997dca96 https://logz.io/blog/istio-instrumenting-microservices-distributed-tracing/ https://opensearch.org/blog/update/2021/06/opensearch-release-candidate-announcement/  https://thenewstack.io/tracing-why-logs-arent-enough-to-debug-your-microservices/  https://www.theregister.com/2021/06/29/kubernetes_spend_report/ Socials: Twitter:⁠ https://twitter.com/OpenObserv⁠ YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠

OpenSearch project was born out of the passion for Elasticsearch and Kibana and the desire to keep them open source in the face of Elastic’s decision to close-source them. After a couple of months of hard work led by AWS, the Beta release was announced earlier this month under Apache2 license. On this episode of OpenObservability Talks I hosted Kyle Davis, Senior Developer Advocate for OpenSearch at AWS. We talked about how OpenSearch came to be, what it took to fork Elasticsearch and Kibana, what the engineers discovered when they dug into the code, what’s planned ahead, and much more. About Kyle Davis: While being a relative newcomer to Amazon, Kyle has a long history with software development and databases. When not working, Kyle enjoys 3D printing, and getting his hand dirty in his Edmonton, Alberta-based home garden. The episode was live-streamed on 27 May 2021 and the video is available at https://youtube.com/live/UDvWdTeH5V4 Resources: https://github.com/opensearch-project Beta announcement Roadmap available Put the OPEN in Observability: Elasticsearch and Kibana relicensing and community chat - OpenObservability Talks S1E08 Socials: Twitter:⁠ https://twitter.com/OpenObserv⁠ YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠

We are thrilled to have Juraci Kröhling a Software Engineer at Red Hat; CNCF, Maintainer for Jaeger, and OpenTelemetry. He will be live and in-person this month on the podcast in a discussion with Jonah Kowall who is the CTO at logz.io and contributor to Jaeger, OpenTelemetry, and OpenSearch.

Join Jonah Kowall and Bartek Plotka for a discussion on the latest happening topics on open source observability. Bartek works on many projects in open source and is Principal Software Engineer at Red Hat; CNCF SIG Observability Tech Lead. He is very active in the community as one of the leaders of Prometheus, Thanos, OpenMetrics, and many other projects.

The ninth of our OpenObservability Talks has Jujhar Singh, Global DevSecOps Practice Lead at The Economist. How much observability is enough? What is the investment required to achieve it? How can we drive observability in the company in a measured and pragmatic way?  This was first live-streamed on 25 February 2021 and the video is available at https://youtu.be/sR-Q3Z-YP2E Show Notes: How to drive observability in your organization What is the minimum observability needed for your organization? The tech stack impact on observability needs Direct correlation between organization, product and observability How to assess your observability needs The investment involved in observability eBPF and tools for deep Linux inspection OpenSearch (Elasticsearch fork) status update OpenTelemetry’s Tracing specification reaches v1.0 Stanza contributed its logging agent to OpenTelemetry Docker was contributed to CNCF Resources: https://www.kiwico.com/ https://sre.google/sre-book/monitoring-distributed-systems/ https://devsecops.jujhar.com/observability-strategy/ https://devsecops.jujhar.com Elasticsearch/Kibana fork updates: https://discuss.opendistrocommunity.dev/c/forking-elasticsearch-kibana/50 OpenTelemetry v1.0 for Tracing: https://medium.com/opentelemetry/opentelemetry-specification-v1-0-0-tracing-edition-72dd08936978 Donating Docker Distribution to the CNCF: https://www.docker.com/blog/donating-docker-distribution-to-the-cncf/ Socials: Twitter: https://twitter.com/OpenObserv Twitch: https://www.twitch.tv/openobservability YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

The eighth of our OpenObservability Talks has Tomer Levy, CEO & Founder of Logz.io. The community is in turmoil around Elastic's announced plan to take Elasticsearch and Kibana off open source. In this episode, both Dotan and Mike have the pleasure of hosting Tomer where we discuss the recent news of Elastic moving Elasticsearch and Kibana to a dual non-OSS license - SSPL and Elastic License - and the implications that have on the open source community around it, including plans to fork Elasticsearch and Kibana, AWS announcement and more. We also talk about what Logz.io hopes to do, and how it wants the OSS to be better than ever. Tomer Levy is co-founder and CEO of Logz.io. Before founding Logz.io, Tomer was the co-founder and CTO of Intigua, and prior to that he managed the Intrusion Prevention System at CheckPoint. Tomer has an M.B.A. from Tel Aviv University and a B.S. in computer science and is an enthusiastic kitesurfer. The live streaming of the OpenObservability Talks is on the last Thursday of each month, and you can join us on Twitch or YouTube Live. Socials: Website: https://openobservability.io/   Twitter: https://twitter.com/OpenObserv   Twitch: https://www.twitch.tv/openobservability   YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

The seventh of our OpenObservability Talks has Andy Thurai, Senior Analyst at GigaOM. Jonah Kowall, CTO and Andy Thurai will talk about Andy's career and journey as a vendor and an analyst. We will discuss the observability market along with APM and other aspects of monitoring. We will then dive into the open-source ecosystem and how this is changing vendor thinking. Of course, we'll also be discussing OpenTelemetry! Andy Thurai is the Founder & Principal at the FieldCTO providing content and advisory services to enterprise customers in particular on AIOps, CloudOps, AI, ML, and Observability areas. He is an accomplished IT executive, strategist, advisor and evangelist with 25+ years of experience in executive, technical and architectural leadership positions at companies such as IBM, Intel, BMC, Nortel and Oracle; he advises many start-ups, and he is a Steering Committee Member for AIOps Exchange. He has been a keynote speaker in many major conferences, as well as a host of many webcasts, podcasts and video chats. He is a regular Forbes contributor and has written 100+ articles on emerging technology topics for publications such as Forbes, AI World, VentureBeat and Wired. Andy Thurai can be reached on Twitter at @AndyThurai, or on LinkedIn. This was first streamed at https://www.twitch.tv/openobservability on December 29th and the full video is available at https://youtu.be/Hr4lGqLiMa0 The live streaming of the OpenObservability Talks is on the last Thursday of each month, and you can join us on Twitch or YouTube Live. Socials: Website: https://openobservability.io/ Twitter: https://twitter.com/OpenObserv Twitch: https://www.twitch.tv/openobservability YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

The sixth of our OpenObservability Talks has Avishai Ish-Shalom, Developer Advocate at ScyllaDB. We trust our metrics to show us the status of our system and where it misbehaves. But do our metrics show us what really happened? You'd be surprised how often it's not the case.  On this episode we discussed the math behind metrics, some common misconceptions, what it take to have accurate metrics, and if there even is such a thing. Avishai Ish-Shalom has served as Engineer in Residence in Aleph VC, engineering manager at Wix.com, co-founded Fewbytes and consulted many other companies on software operations, reliability, design and culture. Currently Avishai is a Developer Advocate for ScyllaDB, the open source NoSQL database. This was first streamed at https://www.twitch.tv/openobservability on November 26th and the full video is available at https://www.youtube.com/watch?v=t9hpWv7fVSk  The live streaming of the OpenObservability Talks is on the last Thursday of each month, and you can join us on Twitch or YouTube Live. Socials: Website: https://openobservability.io/ Twitter: https://twitter.com/OpenObserv Twitch: https://www.twitch.tv/openobservability YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg Links Shared in the recording: DevOps Pulse 2020 Engineer's guide to data analysis  Mature optimization handbook The art of monitoring SRE books