From 070356c9a013c3ea975494d82e4fc7f3ca18b21c Mon Sep 17 00:00:00 2001 From: Anna Scholtz Date: Thu, 12 Feb 2026 13:39:05 -0800 Subject: [PATCH 1/2] Document common OpMon use cases --- src/cookbooks/operational_monitoring.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/src/cookbooks/operational_monitoring.md b/src/cookbooks/operational_monitoring.md index 1b40075ee..1919b851f 100644 --- a/src/cookbooks/operational_monitoring.md +++ b/src/cookbooks/operational_monitoring.md @@ -460,6 +460,22 @@ The table `projects_v1` in `operational_monitoring_derived` contains metadata ab OpMon is updated nightly by telemetry-airflow. It is invoked by the [operational_monitoring DAG](https://github.com/mozilla/telemetry-airflow/blob/main/dags/operational_monitoring.py). +## Common Use Cases + +- **Rollout Monitoring:** Monitor rollouts to detect regressions before they affect the entire population. Compares enabled vs disabled branches for search metrics, performance, and error rates. The rollout monitoring dashboards get created automatically. +- **Build-over-Build Performance:** Detect performance regressions between consecutive builds on pre-release channels (Beta, Nightly). Tracks page load, FCP, input latency, JS execution, and memory metrics. +- **Product Health Dashboards:** Continuous monitoring of stability metrics without end dates: crash rates, hangs, exceptions, success/failure rates. +- **DAU/MAU Tracking and other Business Metrics:** Aggregate-level monitoring of daily/monthly active users, often with KPI forecasting to detect retention issues. Track revenue-impacting metrics like search monetization, subscription engagement, and customer satisfaction. +- **Experiments & A/B Tests:** As an addition to Experimenter, OpMon can be used to track experiment-specific success criteria and side effects for earlier signals. +- **Background Processes:** Monitor Firefox's background services (Background Update, Default Browser Agent). Tracks success rates, exception volumes, and timing metrics, often segmented by OS version. + +## When to use OpMon? + +- OpMon → Continuous monitoring, rollouts, build-over-build comparisons +- Experimenter → A/B tests requiring statistical significance testing +- GLAM → Ad-hoc histogram exploration +- Custom SQL → Complex queries beyond OpMon's aggregation model + ## Experiments vs OpMon The requirements for Operational Monitoring are related to, but mostly distinct from those for experiments: From 16b24a23343a8f8e5fb9646674c1d833f354420c Mon Sep 17 00:00:00 2001 From: Anna Scholtz Date: Thu, 12 Feb 2026 13:41:21 -0800 Subject: [PATCH 2/2] Fix experimenter links --- src/concepts/metric_hub.md | 4 ++-- src/cookbooks/operational_monitoring.md | 2 +- src/datasets/jetstream.md | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/concepts/metric_hub.md b/src/concepts/metric_hub.md index 4f0f06973..38aeb3e92 100644 --- a/src/concepts/metric_hub.md +++ b/src/concepts/metric_hub.md @@ -2,7 +2,7 @@ Metric Hub is a [repository](https://github.com/mozilla/metric-hub) that contains metric, data source and segment definitions that have been reviewed and can be seen as the source of truth. -Definitions that are part of Metric Hub can be referenced in configurations of other tooling as well, such as [Jetstream](https://experimenter.info/deep-dives/jetstream/overview) and [OpMon](https://docs.telemetry.mozilla.org/cookbooks/operational_monitoring.html?highlight=opmon#operational-monitoring-opmon). +Definitions that are part of Metric Hub can be referenced in configurations of other tooling as well, such as [Jetstream](https://experimenter.info/data-analysis/jetstream/overview) and [OpMon](https://docs.telemetry.mozilla.org/cookbooks/operational_monitoring.html?highlight=opmon#operational-monitoring-opmon). Metric Hub is a "thin" metric layer that lies between the Data Warehouse and consumers, such as analysis tools. The Data Warehouse contains the datasets and tables referenced in the metric definitions. Analysis tools can reference metrics that are defined in metric-hub. @@ -256,7 +256,7 @@ Segment SQL snippets need to be boolean expressions to be valid. All the definitions are automatically available in some of our tooling: -- [Jetstream](https://experimenter.info/deep-dives/jetstream/overview) - used for analyzing experiments +- [Jetstream](https://experimenter.info/data-analysis/jetstream/overview) - used for analyzing experiments - [mozanalysis](https://github.com/mozilla/mozanalysis) - a Python library which standardizes how experiment data is analyzed at Mozilla - [OpMon](https://docs.telemetry.mozilla.org/cookbooks/operational_monitoring.html) - a tool for monitoring operational metrics - [bigquery-etl](https://github.com/mozilla/bigquery-etl) - used for writing ETL queries diff --git a/src/cookbooks/operational_monitoring.md b/src/cookbooks/operational_monitoring.md index 1919b851f..0c9ed265f 100644 --- a/src/cookbooks/operational_monitoring.md +++ b/src/cookbooks/operational_monitoring.md @@ -17,7 +17,7 @@ Access to the Looker Operational Monitoring dashboards is currently limited to M ## Configuring a Operational Monitoring project To add or update a project configuration, open a pull request against the `opmon/` directory in [metric-hub](https://github.com/mozilla/metric-hub/tree/main/opmon). -Consider using and adding metric definitions to the [metric-hub](https://github.com/mozilla/metric-hub) that should be available for use across other tools, such as [Jetstream](https://experimenter.info/deep-dives/jetstream/overview), as well. +Consider using and adding metric definitions to the [metric-hub](https://github.com/mozilla/metric-hub) that should be available for use across other tools, such as [Jetstream](https://experimenter.info/data-analysis/jetstream/overview), as well. CI checks will validate the columns, data sources, and SQL syntax. Once CI completes, the pull request gets automatically approved and can be merged. Results for the new project will be available within the next 24 hours. diff --git a/src/datasets/jetstream.md b/src/datasets/jetstream.md index 38851741f..c3b269dc1 100644 --- a/src/datasets/jetstream.md +++ b/src/datasets/jetstream.md @@ -211,4 +211,4 @@ by interacting with the [`jetstream-config`] repository. ## Documentation -Additional documentation about Jetstream can be found in the [Jetstream Experimenter Docs](https://experimenter.info/deep-dives/jetstream/overview). +Additional documentation about Jetstream can be found in the [Jetstream Experimenter Docs](https://experimenter.info/data-analysis/jetstream/overview).