Rob Skillington on Metrics Collection, Uber’s M3, and OpenMetrics

In this podcast, Rob Skillington, co-founder and CTO at Chronosphere, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: metrics collection at scale, multi-dimensional metrics and high-cardinality, developer experience with platform tooling, and open standards related to observability. Why listen to this podcast: - Over the past ten years the requirements related to monitoring and alerting, and the approach taken to implement this, has changed considerably. Compute is now ephemeral and dynamic, services are more numerous, and engineers want to instrument more things. Scalability of a monitoring solution is vitally important. - One of the challenges with metric data is the limited information for providing context for collected values. This can be solved by using multi-dimensional metrics. Dimensions of a metric are name-value pairs that carry additional data to describe the metric value. High dimensionality can lead to high cardinality. - Uber’s M3 metrics collection system initially used open source components such as Cassandra and ElasticSearch for storage and indexing. As the scale of usage of M3 increased, these OSS components were gradually replaced by custom components, such as M3DB. - Building an effective user experience for operational tooling, especially observability-foused tooling, is vitally important. Engineers will be interacting with these tools on a daily basis. They will also be relying on these tools for both alerting and being able to locate and understand what is occurring during production issues. - Open standards are vitally important for interoperability. The OpenMetrics project is an effort to create an open standard for transmitting metrics at scale, with support for both text representation and protocol buffers. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3i3dMPO You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/3i3dMPO