RFC on Alert / SLO label names

[emschwartz]

In autometrics-dev/autometrics-rs#42, we changed how we do alerts such that we have a single alert definition file and use labels to enable the otherwise-dormant recording and alerting rules.

The labels we are currently using are:

• slo_name - user-specified identifier of the SLO (could also be something like objective or objective_name instead)
• objective - percent, expressed as a decimal, that must succeed in order for the SLO to be met (could also be objective_quantile, objective_fraction, or something else)
• target_latency - for latency-related SLOs, the latency cutoff, specified in seconds

What do you think? Should we change any of these? Better to do it now before it would be a breaking change.

#beginbikeshedding

[flenter]

The naming convention here is leaning towards open telemetry, right? I guess whatever is decided in the OpenTelemetry or OpenMetrics semantic conventions discussion will change the naming here. Alternatively we could change the label names to be single word only but I don't think that's a great idea (target_latency in my opinion is better than simply latency )

[emschwartz]

Ah, right. Good point! We actually should namespace these. So it's more like objective.name, objective.quantile, objective.latency

[gagbo]

My 2 cents on the names:

• slo_name feels too technical/implementation oriented, service_name seems more descriptive to me, as it's supposed to be the name of the service for which you defined Objectives
• I prefer threshold to objective because that makes me think that something is going to happen when the metric crosses the threshold (but that might just be me not being a native speaker)
• target_latency is cool, could bikeshed that to max_allowed_latency, but doesn't really matter what the name is. What might be important though is to have the unit in the label name, since OT/OM use different units for time based metrics, it might be important to add a suffix that specifies the unit at all times to help everyone remember what the unit will be, without having to go through docs

[emschwartz]

service_name seems more descriptive to me, as it's supposed to be the name of the service for which you defined Objectives

Hmm, I'm not sure that these are strictly at the level of a service, despite the word service being in SLO. With autometrics you could define objectives for your API handlers separately from your database functions, for example.

I prefer threshold to objective because that makes me think that something is going to happen when the metric crosses the threshold

The main reason I didn't go with threshold initially is that in a way there might be two types of "thresholds" at play. One is the percentage of requests, but in the latency case, that latency cutoff is also another kind of threshold.

What might be important though is to have the unit in the label name, since OT/OM use different units for time based metrics

Interesting -- had to think about this one for a bit. I think we should not add the unit in the label, because this label is already going to be attached to a metric name that either should have the unit in it or we'll need to pick one of the conventions. The unit of this label should match the unit of the metric itself

[gagbo]

Fair enough, it's really not a hill I'm willing to die on anyways :)

[emschwartz]

Based on the input from @flenter and @gagbo, I think we should go with:

• objective.name
• objective.percentile (written as 99 instead of 0.99)
• objective.latency_threshold (only applies to the function.calls.duration metric)

[IvanMerrill]

I think these names make sense. Units should definitely be included in metric names, not labels.

I like using the objective namespace to group them all together.

I think threshold makes more sense to people especially when we're talking in an alerting context.

I think using objective is better than something like SLO because it's simpler and less scary for people. Plus as Evan states, these aren't necessarily service objectives.