Charity Majors @mipsytipsy CTO @honeycombio, ex-Parse, Facebook, Linden Lab; cowrote Database Reliability Engineering; loves whiskey, rainbows. I test in production and so do you. 🌈🖤 Jun. 04, 2019 2 min read

i'm feeling inspired seeing all these tweets and comments flooding out of #monitorama.

so i'm going to tweet some factoids and common misunderstandings throughout the day as they come to me (or as i notice people committing said sins) #o11yfacts incoming!

#o11yfacts 1: there are two different meanings of the word 'metrics'.

There are small-m "metrics", a generic collective noun for statistical data. And there are statsd "Metrics", which are a specific data type consisting of a single number with n tags appended to it.

Tags are a way of adding dimensions to the number so they can be sorted, grouped and described more usefully.

Statsd Metrics can be one of a small number of types, e.g. counter, gauge, histogram, summary, timing, etc, which inform the aggregator how to process or bucket them.

For a long time statsd and its descendants (wavefront, signalfx, datadog, etc) have been the dominant game in town when it comes to monitoring.

This means a lot of people have the Metrics data model lodged firmly in their heads, and assume that Metrics and metrics are the same.

Metrics are fast, cheap, small, have a fixed schema, and are a great fit for tsdbs -- you can pre-aggregate before they even reach the server, then downsample and lose resolution over time so you retain long term data without increasing your storage footprint or cost.

But Metrics have severe limitations, too.

You might fire off hundreds of Metrics over the course of a single request, yet they are disconnected from one another; there is no way of guaranteeing that any two of them describe the same request or were true at the same time.

If you take advantages of the properties that make them cheap, you quickly find yourself in the realm of nonsense numbers that make no god damn sense, because they're averages of averages (of averages).

And statsd descendants only handle low cardinality dimensions.

Therefore, Metrics-based systems are primarily usable for monitoring known-unknowns or describing the health of the system in aggregate over time.

They are terrible for debugging scenarios, or trying to understand user experience, or unknown-unknowns. Just about impossible.

Lots of people use metrics/Metrics interchangeably. I never say it unless I'm referring to statsd-style Metrics, to avoid confusion.

As you investigate observability and adopt newer paradigms for complex systems, you may want to familiarize yourself with this difference. 🍻


You can follow @mipsytipsy.



Bookmark

____
Tip: mention @threader_app on a Twitter thread with the keyword “compile” to get a link to it.

Enjoy Threader? Sign up.

Threader is an independent project created by only two developers. The site gets 500,000+ visits a month and our iOS Twitter client was featured as an App of the Day by Apple. Running this space is expensive and time consuming. If you find Threader useful, please consider supporting us to make it a sustainable project.