diff --git a/docs/specs/om/open_metrics_spec_2_0.md b/docs/specs/om/open_metrics_spec_2_0.md index 3717346c7..d89bd59f6 100644 --- a/docs/specs/om/open_metrics_spec_2_0.md +++ b/docs/specs/om/open_metrics_spec_2_0.md @@ -62,7 +62,16 @@ This section MUST be read together with the ABNF section. In case of disagreemen #### Values -Metric values in OpenMetrics MUST be either floating points or integers. Note that ingestors of the format MAY only support float64. The non-real values NaN, +Inf and -Inf MUST be supported. NaN MUST NOT be considered a missing value, but it MAY be used to signal a division by zero. +Metric values in OpenMetrics MUST be either numbers or complex data types. + +Numbers MUST be either floating points or integers. Note that ingestors of the format MAY only support float64. The non-real values NaN, +Inf and -Inf MUST be supported. NaN MUST NOT be considered a missing value, but it MAY be used to signal a division by zero. + +Complex data types MUST contain all information necessary to recreate a sample of a Metric Type, with the exception of Created Value and Exemplars. + +List of complex data types: + +- Integer counter classic histogram for the Metric Type Histogram. +- Classic summary for the Metric Type Summary. ##### Booleans @@ -325,6 +334,7 @@ metric-type = counter / gauge / histogram / gaugehistogram / stateset metric-type =/ info / summary / unknown sample = metricname [labels] SP number [SP timestamp] [SP created] [exemplar] LF +sample =/ metricname [labels] SP "{" complextype "}" [SP timestamp] [SP created] *exemplar LF exemplar = SP HASH SP labels SP number [SP timestamp] @@ -345,6 +355,13 @@ realnumber = [SIGN] 1*DIGIT realnumber =/ [SIGN] 1*DIGIT ["." *DIGIT] [ "e" [SIGN] 1*DIGIT ] realnumber =/ [SIGN] *DIGIT "." 1*DIGIT [ "e" [SIGN] 1*DIGIT ] +; Quantile is a real number between 0 and 1 inclusive. +quantile = 1*"0" ["." *DIGIT] +quantile =/ *"0" "1" ["." *"0"] + +; Integer counters for complex types. +; Leading 0s explicitly okay. +non-negative-integer = DIGIT *DIGIT ; RFC 5234 is case insensitive. ; Uppercase @@ -387,6 +404,34 @@ escaped-char =/ BS normal-char ; Any unicode character, except newline, double quote, and backslash normal-char = %x00-09 / %x0B-21 / %x23-5B / %x5D-D7FF / %xE000-10FFFF +; Complex types +complextype = classic-histogram / classic-summary + +; count:12,sum:100.0,bucket:[0.1:3,05:12,+Inf:12] +classic-histogram = ch-count "," ch-sum "," ch-bucket + +; count:x where x is a real number, not +-Inf or NaN +ch-count = %d99.111.117.110.116 ":" non-negative-integer +; sum:x where x is a real number or +-Inf or NaN +ch-sum = %d115.117.109 ":" number +; bucket:[...] +ch-bucket = %d98.117.99.107.101.116 ":" "[" ch-le-counts "]" +ch-le-counts = ch-pos-inf-bucket / (ch-neg-inf-bucket / ch-le-bucket) *("," ch-le-bucket) "," ch-pos-inf-bucket +ch-pos-inf-bucket = "+" %d73.110.102 ":" non-negative-integer +ch-neg-inf-bucket = "-" %d73.110.102 ":" non-negative-integer +ch-le-bucket = realnumber ":" non-negative-integer + +; count:12.0,sum:100.0,quantile:[0.9:20.0,0.95:18.0,0.99:10.0] +classic-summary = cs-count "," cs-sum "," cs-quantile + +; count:x where x is a real non negative integer number, not +-Inf or NaN +cs-count = %d99.111.117.110.116 ":" non-negative-integer +; sum:x where x is a real number or +-Inf or NaN +cs-sum = %d115.117.109 ":" number +; quantile:[...] +cs-quantile = %d113.117.97.110.116.105.108.101 ":" "[" cs-q-values "]" +cs-q-values = cs-q-value *("," cs-q-value) +cs-q-value = quantile ":" realnumber ; Lowercase ct @ timestamp created = %d99.116 "@" timestamp ``` @@ -406,13 +451,16 @@ Line endings MUST be signalled with line feed (\n) and MUST NOT contain carriage An example of a complete exposition: ```openmetrics +# TYPE acme_http_server_request_seconds histogram +# UNIT acme_http_server_request_seconds seconds +# HELP acme_http_server_request_seconds Latency though all of ACME's HTTP request service. +acme_http_router_request_seconds{path="/api/v1",method="GET"} {count:10,sum:100.0,bucket:[0.1:2,0.5:10,+Inf:10]} +acme_http_router_request_seconds{path="/api/v2",method="GET"} {count:1,sum:10.0,bucket:[0.1:1,0.5:1,+Inf:1]} # TYPE acme_http_router_request_seconds summary # UNIT acme_http_router_request_seconds seconds # HELP acme_http_router_request_seconds Latency though all of ACME's HTTP request router. -acme_http_router_request_seconds_sum{path="/api/v1",method="GET"} 9036.32 ct@1605281325.0 -acme_http_router_request_seconds_count{path="/api/v1",method="GET"} 807283.0 ct@1605281325.0 -acme_http_router_request_seconds_sum{path="/api/v2",method="POST"} 479.3 ct@1605301325.0 -acme_http_router_request_seconds_count{path="/api/v2",method="POST"} 34.0 ct@1605301325.0 +acme_http_router_request_seconds{path="/api/v1",method="GET"} {count:10,sum:100.0,quantile:[0.95:20.0,0.99:10.2]} ct@1605281325.0 +acme_http_router_request_seconds{path="/api/v2",method="GET"} {count:1,sum:10.0,quantile:[0.95:15.,0.99:8.2]} ct@1605301325.0 # TYPE go_goroutines gauge # HELP go_goroutines Number of goroutines that currently exist. go_goroutines 69 @@ -440,28 +488,6 @@ Integer numbers MUST NOT have a decimal point. Examples are `23`, `0042`, and `1 Floating point numbers MUST be represented either with a decimal point or using scientific notation. Examples are `8903.123421` and `1.89e-7`. Floating point numbers MUST fit within the range of a 64-bit floating point value as defined by IEEE 754, but MAY require so many bits in the mantissa that results in lost precision. This MAY be used to encode nanosecond resolution timestamps. -Arbitrary integer and floating point rendering of numbers MUST NOT be used for "quantile" and "le" label values as in section "Canonical Numbers". They MAY be used anywhere else numbers are used. - -###### Considerations: Canonical Numbers - -Numbers in the "le" label values of histograms and "quantile" label values of summary metrics are special in that they're label values, and label values are intended to be opaque. As end users will likely directly interact with these string values, and as many monitoring systems lack the ability to deal with them as first-class numbers, it would be beneficial if a given number had the exact same text representation. - -Consistency is highly desirable, but real world implementations of languages and their runtimes make mandating this impractical. The most important common quantiles are 0.5, 0.95, 0.9, 0.99, 0.999 and bucket values representing values from a millisecond up to 10.0 seconds, because those cover cases like latency SLAs and Apdex for typical web services. Powers of ten are covered to try to ensure that the switch between fixed point and exponential rendering is consistent as this varies across runtimes. The target rendering is equivalent to the default Go rendering of float64 values (i.e. %g), with a .0 appended in case there is no decimal point or exponent to make clear that they are floats. - -Exposers MUST produce output for positive infinity as +Inf. - -Exposers SHOULD produce output for the values 0.0 up to 10.0 in 0.001 increments in line with the following examples: -0.0 0.001 0.002 0.01 0.1 0.9 0.95 0.99 0.999 1.0 1.7 10.0 - -Exposers SHOULD produce output for the values 1e-10 up to 1e+10 in powers of ten in line with the following examples: -1e-10 1e-09 1e-05 0.0001 0.1 1.0 100000.0 1e+06 1e+10 - -Parsers MUST NOT reject inputs which are outside of the canonical values merely because they are not consistent with the canonical values. For example 1.1e-4 must not be rejected, even though it is not the consistent rendering of 0.00011. - -Exposers SHOULD follow these patterns for non-canonical numbers, and the intention is by adjusting the rendering algorithm to be consistent for these values that the vast majority of other values will also have consistent rendering. Exposers using only a few particular le/quantile values could also hardcode. In languages such as C where a minimal floating point rendering algorithm such as Grisu3 is not readily available, exposers MAY use a different rendering. - -A warning to implementers in C and other languages that share its printf implementation: The standard precision of %f, %e and %g is only six significant digits. 17 significant digits are required for full precision, e.g. `printf("%.17g", d)`. - ##### Timestamps Timestamps SHOULD NOT use exponential float rendering for timestamps if nanosecond precision is needed as rendering of a float64 does not have sufficient precision, e.g. `1604676851.123456789`. @@ -547,44 +573,7 @@ The rendering of values for a MetricPoint can include additional labels (e.g. th #### MetricPoint -MetricPoints MUST NOT be interleaved. - -A correct example where there were multiple MetricPoints and Samples within a MetricFamily would be: - -``` -# TYPE foo_seconds summary -# UNIT foo_seconds seconds -foo_seconds_count{a="bb"} 0 123 -foo_seconds_sum{a="bb"} 0 123 -foo_seconds_count{a="bb"} 0 456 -foo_seconds_sum{a="bb"} 0 456 -foo_seconds_count{a="ccc"} 0 123 -foo_seconds_sum{a="ccc"} 0 123 -foo_seconds_count{a="ccc"} 0 456 -foo_seconds_sum{a="ccc"} 0 456 -``` - -An incorrect example where Metrics are interleaved: - -``` -# TYPE foo_seconds summary -# UNIT foo_seconds seconds -foo_seconds_count{a="bb"} 0 123 -foo_seconds_count{a="ccc"} 0 123 -foo_seconds_count{a="bb"} 0 456 -foo_seconds_count{a="ccc"} 0 456 -``` - -An incorrect example where MetricPoints are interleaved: - -``` -# TYPE foo_seconds summary -# UNIT foo_seconds seconds -foo_seconds_count{a="bb"} 0 123 -foo_seconds_count{a="bb"} 0 456 -foo_seconds_sum{a="bb"} 0 123 -foo_seconds_sum{a="bb"} 0 456 -``` +MetricPoints MUST be contained in a single line. #### Metric types @@ -726,55 +715,28 @@ Metric labels and MetricPoint value labels MAY be in any order. ##### Summary -If present, the MetricPoint's Sum Value Sample MetricName MUST have the suffix `_sum`. If present, the MetricPoint's Count Value MetricName MUST have the suffix `_count`. If present, the MetricPoint's Quantile Values MUST specify the quantile measured using a label with a label name of "quantile" and with a label value of the quantile measured. +The MetricPoint MUST be formatted as the classic summary complex type. -If present the MetricPoint's Created Timestamp MUST be inlined with the Metric point with a `ct@` prefix. If the value's timestamp is present, the Created Timestamp MUST be added right after it. If exemplar is present, the Created Timestamp MUST be added before it. Created Timestamp MUST be appended to all Quantile Values, to the MetricPoint's Sum and MetricPoint's Count. +If present the MetricPoint's Created Timestamp MUST be inlined with the Metric point with a `ct@` prefix. If the value's timestamp is present, the Created Timestamp MUST be added right after it. If exemplar is present, the Created Timestamp MUST be added before it. -An example of a Metric with no labels and a MetricPoint with Sum, Count and Created Timestamp values: +An example of a Metric with no labels and a MetricPoint with Sum, Count, Quantiles, and Created Timestamp values: ```openmetrics-add-eof # TYPE foo summary -foo_count 17.0 ct@1520430000.123 -foo_sum 324789.3 ct@1520430000.123 +foo {count:17,sum:324789.3,quantile:[0.95:5000.0,0.99:1000.0]} ct@1520430000.123 ``` -An example of a Metric with no labels and a MetricPoint with two quantiles and Created Timestamp values: - -```openmetrics-add-eof -# TYPE foo summary -foo{quantile="0.95"} 123.7 ct@1520430000.123 -foo{quantile="0.99"} 150.0 ct@1520430000.123 -``` - -Quantiles MAY be in any order. - ##### Histogram -The MetricPoint's Bucket Values Sample MetricNames MUST have the suffix `_bucket`. If present, the MetricPoint's Sum Value Sample MetricName MUST have the suffix `_sum`. +The MetricPoint MUST be formatted as the integer counter classic histogram complex type. If present the MetricPoint's Created Timestamp MUST be inlined with the Metric point with a `ct@` prefix. If the value's timestamp is present, the Created Timestamp MUST be added right after it. If exemplar is present, the Created Timestamp MUST be added before it. Created Timestamp MUST be appended to all Bucket Values, to the MetricPoint's Sum and MetricPoint's Count. -If and only if a Sum Value is present in a MetricPoint, then the MetricPoint's +Inf Bucket value MUST also appear in a Sample with a MetricName with the suffix "_count". - -Buckets MUST be sorted in number increasing order of "le", and the value of the "le" label MUST follow the rules for Canonical Numbers. - An example of a Metric with no labels and a MetricPoint with Sum, Count, and Created Timestamp values, and with 12 buckets. A wide and atypical but valid variety of “le” values is shown on purpose: ```openmetrics-add-eof # TYPE foo histogram -foo_bucket{le="0.0"} 0 ct@1520430000.123 -foo_bucket{le="1e-05"} 0 ct@1521430000.123 -foo_bucket{le="0.0001"} 5 ct@1521430020.123 -foo_bucket{le="0.1"} 8 ct@1520430321.123 -foo_bucket{le="1.0"} 10 ct@1522430000.123 -foo_bucket{le="10.0"} 11 ct@1520430123.123 -foo_bucket{le="100000.0"} 11 ct@1521430010.123 -foo_bucket{le="1e+06"} 15 ct@1520430301.123 -foo_bucket{le="1e+23"} 16 ct@1521430001.123 -foo_bucket{le="1.1e+23"} 17 ct@1522430220.123 -foo_bucket{le="+Inf"} 17 ct@1520430000.123 -foo_count 17 ct@1520430000.123 -foo_sum 324789.3 ct@1520430000.123 +foo {count:17,sum:324789.3,bucket:[0.0:0,1e-05:0,0.0001:5,0.1:8,1.0:10,10.0:11,100000.0:11,1e+06:15,1e+23:16,1.1e+23:17,+Inf:17]} ct@1520430000.123 ``` ###### Exemplars @@ -786,33 +748,18 @@ The "0.01" bucket has no Exemplar. The 0.1 bucket has an Exemplar with no Labels ```openmetrics-add-eof # TYPE foo histogram -foo_bucket{le="0.01"} 0 ct@1520430000.123 -foo_bucket{le="0.1"} 8 ct@1520430000.123 # {} 0.054 -foo_bucket{le="1"} 11 ct@1520430000.123 # {trace_id="KOO5S4vxi0o"} 0.67 -foo_bucket{le="10"} 17 ct@1520430000.123 # {trace_id="oHg5SJYRHA0"} 9.8 1520879607.789 -foo_bucket{le="+Inf"} 17 ct@1520430000.123 -foo_count 17 ct@1520430000.123 -foo_sum 324789.3 ct@1520430000.123 +foo {count:17,sum:324789.3,bucket:[0.01:0,0.1:8,1.0:11,10.0:17,+Inf:17]} ct@1520430000.123 # {} 0.054 1520879607.7 # {trace_id="KOO5S4vxi0o"} 0.67 1520879602.890 # {trace_id="oHg5SJYRHA0"} 9.8 1520879607.789 ``` ##### GaugeHistogram -The MetricPoint's Bucket Values Sample MetricNames MUST have the suffix `_bucket`. If present, the MetricPoint's Sum Value Sample MetricName MUST have the suffix `_gsum`. -If and only if a Sum Value is present in a MetricPoint, then the MetricPoint's +Inf Bucket value MUST also appear in a Sample with a MetricName with the suffix `_gcount`. - -Buckets MUST be sorted in number increasing order of "le", and the value of the "le" label MUST follow the rules for Canonical Numbers. +The MetricPoint MUST be formatted as the integer counter classic histogram complex type. An example of a Metric with no labels, and one MetricPoint value with no Exemplar with no Exemplars in the buckets: ```openmetrics-add-eof # TYPE foo gaugehistogram -foo_bucket{le="0.01"} 20.0 -foo_bucket{le="0.1"} 25.0 -foo_bucket{le="1"} 34.0 -foo_bucket{le="10"} 34.0 -foo_bucket{le="+Inf"} 42.0 -foo_gcount 42.0 -foo_gsum 3289.3 +foo {count:42,sum:3289.3,bucket:[0.01:20,0.1:25,1:34,+Inf:42]} ``` ##### Unknown