diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 96b8f0a..e4a20ad 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -55,12 +55,14 @@ jobs: - uses: Swatinem/rust-cache@v2 - name: Run rivet docs check run: cargo run --release -p rivet-cli -- docs check - # Subcommand-coverage gate — warn-only initially so the existing - # inventory of uncovered subcommands (variant, baseline, snapshot, - # runs, pipelines, templates, close-gaps) doesn't break the build. - # Flip to `--strict` once those gaps are filled. + # Subcommand-coverage gate — `--warn-only` emits a + # `::warning::` GitHub Actions annotation per uncovered subcommand + # so they surface inline on the PR without failing the build. Flip + # to `--strict` once the existing inventory of uncovered + # subcommands (variant, baseline, snapshot, runs, pipelines, + # templates, close-gaps) is filled. See issue #248 for the design. - name: Subcommand-coverage gate (warn-only) - run: cargo run --release -p rivet-cli -- docs check --coverage + run: cargo run --release -p rivet-cli -- docs check --coverage --warn-only # ── Tests ───────────────────────────────────────────────────────────── test: diff --git a/rivet-cli/src/docs.rs b/rivet-cli/src/docs.rs index 014340e..6973439 100644 --- a/rivet-cli/src/docs.rs +++ b/rivet-cli/src/docs.rs @@ -276,6 +276,15 @@ pub fn has_topic(slug: &str) -> bool { TOPICS.iter().any(|t| t.slug == slug) } +/// Return the raw content body of a topic, or `None` if no topic with +/// this slug is registered. +/// +/// Used by the subcommand-coverage gate's umbrella rule to verify that a +/// parent topic actually mentions the child subcommand by name. +pub fn topic_content(slug: &str) -> Option<&'static str> { + TOPICS.iter().find(|t| t.slug == slug).map(|t| t.content) +} + // ── Embedded documentation ────────────────────────────────────────────── const ARTIFACT_FORMAT_DOC: &str = r#"# Artifact YAML Format @@ -2798,14 +2807,24 @@ reverse direction: every subcommand that EXISTS should be DOCUMENTED. ## Quick start ``` -rivet docs check --coverage # warn-only report -rivet docs check --coverage --strict # fail-on-uncovered (CI gate) +rivet docs check --coverage # local exploration: print, exit 0, no annotations +rivet docs check --coverage --warn-only # CI rollout: print + emit ::warning:: annotations, exit 0 +rivet docs check --coverage --strict # enforcing CI: print, exit 1 on any uncovered rivet docs check --coverage --format json ``` -The default warn-only mode always exits 0 so the gate can land in CI as a -visibility step before the obvious gaps are filled. Once the inventory is -clean, flip `--strict` on to make uncovered subcommands fail the build. +`--warn-only` and `--strict` are mutually exclusive. Pick one for CI +based on rollout phase: + +| Mode | Exit code | GitHub Actions annotations | Use for | +|------|-----------|----------------------------|---------| +| `--coverage` (default) | 0 | none | local exploration | +| `--coverage --warn-only` | 0 | `::warning::` per gap | CI rollout — surface gaps inline on PRs without failing the build | +| `--coverage --strict` | 1 if any gap | none (use `--warn-only` for those) | enforcing CI once the inventory is clean | + +The `::warning file=…::…` lines emitted by `--warn-only` are GitHub +Actions' workflow-command syntax — the runner picks them up from stdout +and renders them as PR review comments without failing the job. ## Coverage rules @@ -2815,14 +2834,22 @@ A subcommand path X (e.g. `schema/show`) is covered if any of: become dashes for slug lookup). 2. The path itself is a top-level subcommand whose name has a topic (e.g. `mcp` is covered by the `mcp` topic). -3. The parent subcommand has a topic (e.g. all `schema/*` paths are - covered by the `schema` parent — currently mapped to `cli`). -4. The subcommand is in the explicit allow-list (built-in commands like +3. The parent subcommand has a topic, found by walking up the path (e.g. + `schema/show` falls back to a `schema` topic when no `schema-show` + topic exists). +4. The top-level subcommand has an entry in `COVERAGE_TOPIC_MAP` AND the + referenced topic body actually mentions the subcommand name as a + whole word (case-insensitive). This rule lets a single umbrella + topic (typically `cli`) cover a family of subcommands — but only if + the topic genuinely documents them. A catch-all entry pointing to a + topic that never references the family is no coverage at all (issue + #248 B5). +5. The subcommand is in the explicit allow-list (built-in commands like `help` that are inherently undocumented). -The mapping is intentionally generous: a single `cli` reference topic -covers every nested action under `schema`, `baseline`, `snapshot`, etc., -provided the parent subcommand itself has a section in `rivet docs cli`. +The body-mention check (rule 4) prevents the umbrella from quietly +papering over real gaps: if `lsp` maps to `cli` but the `cli` topic +body never says `lsp`, the path is reported as uncovered. ## Report format @@ -2848,15 +2875,24 @@ provides coverage (when applicable). ## CI integration +Initial rollout — surface gaps without failing the build: + +```yaml +- name: Subcommand-coverage gate (warn-only) + run: cargo run --release -p rivet-cli -- docs check --coverage --warn-only +``` + +Once the inventory is clean, switch to enforcing: + ```yaml -- name: Subcommand-coverage gate - run: cargo run --release -p rivet-cli -- docs check --coverage - # add --strict once the obvious gaps are filled +- name: Subcommand-coverage gate (strict) + run: cargo run --release -p rivet-cli -- docs check --coverage --strict ``` The CI step is idempotent: it re-derives the subcommand tree from clap's -runtime metadata, so adding a new subcommand without a doc topic will fail -the gate immediately under `--strict`. +runtime metadata, so adding a new subcommand without a doc topic will +fail the gate immediately under `--strict`, or surface as a PR-review +warning under `--warn-only`. ## Allow-list diff --git a/rivet-cli/src/main.rs b/rivet-cli/src/main.rs index f005049..b1f622d 100644 --- a/rivet-cli/src/main.rs +++ b/rivet-cli/src/main.rs @@ -537,8 +537,15 @@ enum Command { #[arg(long)] coverage: bool, + /// (check --coverage only) print a report and exit 0; in addition + /// emit `::warning file=…::…` GitHub Actions annotations for each + /// uncovered subcommand so CI can surface them inline on PRs + /// without failing the build. Mutually exclusive with --strict. + #[arg(long = "warn-only", conflicts_with = "strict")] + warn_only: bool, + /// (check --coverage only) exit non-zero if any subcommand is - /// uncovered. Default is warn-only. + /// uncovered. Default is print-and-exit-0 (no annotations). #[arg(long)] strict: bool, }, @@ -1569,19 +1576,20 @@ fn run(cli: Cli) -> Result { context, fix, coverage, + warn_only, strict, } = &cli.command { if matches!(topic.as_deref(), Some("check")) { if *coverage { - return cmd_docs_coverage(format, *strict); + return cmd_docs_coverage(format, *warn_only, *strict); } return cmd_docs_check(&cli, format, *fix); } // Allow `rivet docs --coverage` (no `check` topic) as a shorthand — // the coverage gate doesn't depend on any other doc-check state. if *coverage { - return cmd_docs_coverage(format, *strict); + return cmd_docs_coverage(format, *warn_only, *strict); } return cmd_docs(topic.as_deref(), *list, grep.as_deref(), format, *context); } @@ -7275,9 +7283,13 @@ fn render_docs_check_json(report: &rivet_core::doc_check::CheckReport) -> String // // See `rivet docs docs-coverage` for the full design. Quick rules: // * A subcommand is covered if its slug (or a parent slug) matches a -// registered topic, OR if the parent's name appears as a topic. +// registered topic, OR if `COVERAGE_TOPIC_MAP` maps it to an umbrella +// topic AND the umbrella topic body actually mentions the child name +// (whole-word, case-insensitive). // * Built-ins like `help` are exempt via `COVERAGE_ALLOWLIST`. -// * Default is warn-only (exit 0); `--strict` makes uncovered fail. +// * Three modes: `--coverage` (print, exit 0), `--coverage --warn-only` +// (print + emit `::warning::` annotations, exit 0), `--coverage +// --strict` (print, exit 1 on any uncovered). /// Top-level subcommands whose docs are inherently unnecessary or are /// surfaced via a different channel. Keep this list short — adding an @@ -7394,11 +7406,17 @@ fn walk_subcommand( /// Compute the coverage rows for a given clap tree and topic-slug set. /// Pulled into its own function so the unit tests can exercise it without /// shelling out to the full CLI. +/// +/// `topic_body` is a lookup for topic bodies — used by the umbrella rule +/// (rule 4) to verify the parent topic actually mentions the child name. +/// In production this is `docs::topic_content`; tests pass a closure over +/// a fake topic registry. fn compute_coverage_rows( root: &clap::Command, topic_slugs: &std::collections::BTreeSet, allow_list: &[&str], topic_map: &[(&str, &str)], + topic_body: &dyn Fn(&str) -> Option<&str>, ) -> Vec { let paths = collect_subcommand_paths(root); let mut rows = Vec::with_capacity(paths.len()); @@ -7411,7 +7429,7 @@ fn compute_coverage_rows( let covered_by = if allow_listed { None } else { - resolve_coverage(&path, top, topic_slugs, topic_map) + resolve_coverage(&path, top, topic_slugs, topic_map, topic_body) }; rows.push(CoverageRow { @@ -7433,11 +7451,15 @@ fn compute_coverage_rows( /// 3. Parent-walk: drop the last segment and retry both separators. /// 4. Manual top-level mapping via `COVERAGE_TOPIC_MAP` — used when a /// single umbrella topic (typically `cli`) documents a family. +/// Tightened in #248: the umbrella topic body must mention the +/// top-level subcommand name as a whole word (case-insensitive), +/// otherwise the path stays uncovered. fn resolve_coverage( path: &str, top: &str, topic_slugs: &std::collections::BTreeSet, topic_map: &[(&str, &str)], + topic_body: &dyn Fn(&str) -> Option<&str>, ) -> Option { // 1. & 2. Exact slug match. if topic_slugs.contains(path) { @@ -7461,18 +7483,54 @@ fn resolve_coverage( } } - // 4. Manual umbrella mapping on the top-level name. + // 4. Manual umbrella mapping on the top-level name. The umbrella + // topic must actually MENTION the child subcommand name — a + // catch-all `cli` mapping that doesn't reference the family is no + // coverage at all (issue #248 B5). for (name, slug) in topic_map { if *name == top && topic_slugs.contains(*slug) { - return Some((*slug).to_string()); + let Some(body) = topic_body(slug) else { + continue; + }; + if topic_body_mentions(body, top) { + return Some((*slug).to_string()); + } } } None } +/// True iff `body` contains `name` as a whole-word, case-insensitive +/// match. Used by the umbrella rule (#248 B5) to ensure a parent topic +/// actually references the child subcommand it claims to cover. +fn topic_body_mentions(body: &str, name: &str) -> bool { + use regex::RegexBuilder; + // Anchor with `\b` so e.g. `query` doesn't match `subquery`. Build + // case-insensitively. `regex::escape` keeps oddly-named subcommands + // (`commit-msg-check`) safe inside the pattern. + let pattern = format!(r"\b{}\b", regex::escape(name)); + RegexBuilder::new(&pattern) + .case_insensitive(true) + .build() + .ok() + .is_some_and(|re| re.is_match(body)) +} + /// Run `rivet docs check --coverage` — assert every subcommand path has /// an embedded doc topic. -fn cmd_docs_coverage(format: &str, strict: bool) -> Result { +/// +/// Three modes (issue #248 B6): +/// * Default (`--coverage`): print the report and exit 0. No annotations. +/// Intended for local exploration. +/// * `--coverage --warn-only`: print + emit one `::warning::` GitHub +/// Actions annotation per uncovered subcommand, exit 0. For staged CI +/// rollout where gaps should surface inline on PRs without failing the +/// build. +/// * `--coverage --strict`: print, exit 1 if anything is uncovered. For +/// enforcing CI once the inventory is clean. +/// +/// `--warn-only` and `--strict` are mutually exclusive (clap-enforced). +fn cmd_docs_coverage(format: &str, warn_only: bool, strict: bool) -> Result { use clap::CommandFactory; use std::collections::BTreeSet; @@ -7480,7 +7538,13 @@ fn cmd_docs_coverage(format: &str, strict: bool) -> Result { let root = Cli::command(); let topic_slugs: BTreeSet = docs::topic_slugs().into_iter().map(String::from).collect(); - let rows = compute_coverage_rows(&root, &topic_slugs, COVERAGE_ALLOWLIST, COVERAGE_TOPIC_MAP); + let rows = compute_coverage_rows( + &root, + &topic_slugs, + COVERAGE_ALLOWLIST, + COVERAGE_TOPIC_MAP, + &|slug| docs::topic_content(slug), + ); let total = rows.iter().filter(|r| !r.allow_listed).count(); // Covered = paths that resolve to a topic (and thus aren't allow-listed). @@ -7495,11 +7559,32 @@ fn cmd_docs_coverage(format: &str, strict: bool) -> Result { _ => print!("{}", render_coverage_text(&rows, total, covered)), } + // Warn-only mode emits GitHub Actions workflow-command annotations so + // CI surfaces every uncovered subcommand inline on the PR. The + // annotations are printed to stdout (not stderr) — that's where the + // GitHub Actions runner scans for `::warning::` lines. + if warn_only && !uncovered.is_empty() { + for row in &uncovered { + // No file path is meaningful here (the gap is in + // docs::TOPICS, not on a YAML line), so attribute to the + // module that owns the registry. + println!( + "::warning file=rivet-cli/src/docs.rs::rivet docs check --coverage: subcommand `{}` is not covered by any topic in docs::TOPICS", + row.path + ); + } + } + let pass = uncovered.is_empty(); if !pass && !strict { eprintln!( - "rivet docs check --coverage: {} subcommand(s) uncovered (warn-only; use --strict to fail)", - uncovered.len() + "rivet docs check --coverage: {} subcommand(s) uncovered{}", + uncovered.len(), + if warn_only { + " (warn-only; emitted ::warning:: annotations; use --strict to fail)" + } else { + " (default mode; use --warn-only for CI annotations or --strict to fail)" + }, ); return Ok(true); } @@ -13445,6 +13530,14 @@ mod coverage_gate_tests { names.iter().map(|s| (*s).to_string()).collect() } + /// Default test body lookup — returns a body that mentions every + /// possible top-level subcommand name, so the umbrella rule (rule 4) + /// fires unconditionally for tests that aren't specifically + /// exercising the body-mention check (#248 B5). + fn permissive_body(_slug: &str) -> Option<&str> { + Some("fruit apple banana grape orange") + } + /// A small fake clap tree: one parent (`fruit`) with two children /// (`apple`, `banana`) — exactly the kind of fixture the /// implementation sketch in the task asked for. @@ -13475,7 +13568,7 @@ mod coverage_gate_tests { fn parent_topic_covers_all_leaves() { let cmd = fake_tree(); let topics = slugs(&["fruit"]); - let rows = compute_coverage_rows(&cmd, &topics, &[], &[]); + let rows = compute_coverage_rows(&cmd, &topics, &[], &[], &permissive_body); // All three rows resolved via the `fruit` topic — the parent // walk is what catches the leaves. assert_eq!(rows.len(), 3); @@ -13491,7 +13584,7 @@ mod coverage_gate_tests { // Both `fruit` AND `fruit-apple` exist — the leaf-specific // topic should win. let topics = slugs(&["fruit", "fruit-apple"]); - let rows = compute_coverage_rows(&cmd, &topics, &[], &[]); + let rows = compute_coverage_rows(&cmd, &topics, &[], &[], &permissive_body); let apple = rows.iter().find(|r| r.path == "fruit/apple").unwrap(); assert_eq!(apple.covered_by.as_deref(), Some("fruit-apple")); let banana = rows.iter().find(|r| r.path == "fruit/banana").unwrap(); @@ -13502,7 +13595,7 @@ mod coverage_gate_tests { fn missing_topic_is_uncovered() { let cmd = fake_tree(); let topics = slugs(&["something-else"]); - let rows = compute_coverage_rows(&cmd, &topics, &[], &[]); + let rows = compute_coverage_rows(&cmd, &topics, &[], &[], &permissive_body); for r in &rows { assert!(!r.is_covered(), "row {r:?} should be uncovered"); assert!(r.covered_by.is_none()); @@ -13513,7 +13606,7 @@ mod coverage_gate_tests { fn allow_list_exempts_path() { let cmd = fake_tree(); let topics = BTreeSet::new(); - let rows = compute_coverage_rows(&cmd, &topics, &["fruit"], &[]); + let rows = compute_coverage_rows(&cmd, &topics, &["fruit"], &[], &permissive_body); for r in &rows { assert!( r.allow_listed, @@ -13524,18 +13617,122 @@ mod coverage_gate_tests { } } + fn body_cli_mentions_fruit(slug: &str) -> Option<&str> { + if slug == "cli" { + Some("This is the CLI reference. It documents fruit and other commands.") + } else { + None + } + } + + fn body_cli_no_fruit(slug: &str) -> Option<&str> { + if slug == "cli" { + Some("This is the CLI reference. It documents validate, list, get.") + } else { + None + } + } + + fn body_cli_fruit_capitalised(slug: &str) -> Option<&str> { + if slug == "cli" { + Some("Fruit reference.") + } else { + None + } + } + + fn body_cli_subquery(slug: &str) -> Option<&str> { + if slug == "cli" { + Some("Run a subquery.") + } else { + None + } + } + #[test] - fn topic_map_provides_umbrella_coverage() { + fn topic_map_provides_umbrella_coverage_when_body_mentions_child() { let cmd = fake_tree(); let topics = slugs(&["cli"]); - // `fruit` -> `cli` umbrella mapping. + // `fruit` -> `cli` umbrella mapping; the `cli` topic body must + // mention `fruit` for the umbrella rule to fire (#248 B5). let map = &[("fruit", "cli")]; - let rows = compute_coverage_rows(&cmd, &topics, &[], map); + let rows = compute_coverage_rows(&cmd, &topics, &[], map, &body_cli_mentions_fruit); for r in &rows { assert_eq!(r.covered_by.as_deref(), Some("cli")); } } + /// #248 B5: the umbrella rule (rule 4) MUST require the parent topic + /// body to mention the child subcommand by name as a whole word. + /// Without this check, a single sloppy `cli` umbrella mapping would + /// claim coverage for every family it lists, even when the body + /// never references them. + #[test] + fn umbrella_rule_rejects_unmentioned_child() { + let cmd = fake_tree(); + let topics = slugs(&["cli"]); + let map = &[("fruit", "cli")]; + // Body talks about other things; `fruit` never appears. + let rows = compute_coverage_rows(&cmd, &topics, &[], map, &body_cli_no_fruit); + for r in &rows { + assert!( + !r.is_covered(), + "without a body mention, umbrella rule must not cover {r:?}", + ); + assert!(r.covered_by.is_none()); + } + } + + /// #248 B5: matching is case-insensitive — a `Fruit`-cased mention + /// in prose still satisfies the rule for child name `fruit`. + #[test] + fn umbrella_body_mention_is_case_insensitive() { + let cmd = fake_tree(); + let topics = slugs(&["cli"]); + let map = &[("fruit", "cli")]; + let rows = compute_coverage_rows(&cmd, &topics, &[], map, &body_cli_fruit_capitalised); + for r in &rows { + assert_eq!(r.covered_by.as_deref(), Some("cli")); + } + } + + /// #248 B5: matching is whole-word — a substring match like + /// `subquery` containing `query` must NOT satisfy the rule for child + /// name `query`. + #[test] + fn umbrella_body_mention_requires_whole_word() { + // Tree with `query` as the only child so we can use it as `top`. + let cmd = clap::Command::new("rivet").subcommand(clap::Command::new("query")); + let topics = slugs(&["cli"]); + let map = &[("query", "cli")]; + let rows = compute_coverage_rows(&cmd, &topics, &[], map, &body_cli_subquery); + let row = rows.iter().find(|r| r.path == "query").unwrap(); + assert!( + !row.is_covered(), + "substring match must not satisfy the umbrella rule", + ); + } + + #[test] + fn topic_body_mentions_word_boundary() { + // Whole-word match: positive cases. + assert!(topic_body_mentions("rivet query foo", "query")); + assert!(topic_body_mentions("Use the QUERY command.", "query")); + assert!(topic_body_mentions( + "commit-msg-check is a hook", + "commit-msg-check" + )); + assert!(topic_body_mentions( + "(query) parens count as boundaries", + "query" + )); + // Whole-word match: negative cases. + assert!(!topic_body_mentions("subquery", "query")); + assert!(!topic_body_mentions("queries are different", "query")); + assert!(!topic_body_mentions("noquery", "query")); + assert!(!topic_body_mentions("", "query")); + } + #[test] fn coverage_slug_replaces_slashes() { assert_eq!(coverage_slug("schema/show"), "schema-show"); diff --git a/rivet-cli/tests/docs_coverage.rs b/rivet-cli/tests/docs_coverage.rs index 7b585b1..ef8c71a 100644 --- a/rivet-cli/tests/docs_coverage.rs +++ b/rivet-cli/tests/docs_coverage.rs @@ -42,11 +42,11 @@ fn rivet_bin() -> std::path::PathBuf { workspace_root.join("target").join("debug").join("rivet") } -/// `rivet docs check --coverage` succeeds (exit 0) by default — warn-only -/// mode is the default contract so the gate can land in CI without -/// breaking on the existing inventory of uncovered commands. +/// `rivet docs check --coverage` (no flag) succeeds (exit 0) and emits +/// the report — but no `::warning::` annotations (those are reserved for +/// `--warn-only`). This is the local-exploration mode (#248 B6). #[test] -fn coverage_warn_only_exits_zero() { +fn coverage_default_exits_zero_no_annotations() { let output = Command::new(rivet_bin()) .args(["docs", "check", "--coverage"]) .output() @@ -54,7 +54,7 @@ fn coverage_warn_only_exits_zero() { assert!( output.status.success(), - "warn-only mode must exit 0; stderr: {}", + "default mode must exit 0; stderr: {}", String::from_utf8_lossy(&output.stderr), ); @@ -79,15 +79,83 @@ fn coverage_warn_only_exits_zero() { "expected '{name}' in coverage output, got:\n{stdout}" ); } + + // Default mode does NOT emit GitHub Actions annotations — those are + // reserved for `--warn-only`. + assert!( + !stdout.contains("::warning"), + "default mode must NOT emit ::warning:: annotations; got:\n{stdout}" + ); +} + +/// `--warn-only` exits 0 AND emits one `::warning file=…::…` GitHub +/// Actions annotation per uncovered subcommand. The annotations let CI +/// surface the gaps inline on the PR without failing the build (#248 B6). +#[test] +fn coverage_warn_only_emits_github_annotations() { + let output = Command::new(rivet_bin()) + .args(["docs", "check", "--coverage", "--warn-only"]) + .output() + .expect("failed to execute rivet docs check --coverage --warn-only"); + + assert!( + output.status.success(), + "--warn-only must exit 0; stderr: {}", + String::from_utf8_lossy(&output.stderr), + ); + + let stdout = String::from_utf8_lossy(&output.stdout); + + // The current rivet repo has known uncovered subcommands (variant, + // baseline, snapshot, runs, pipelines, templates, close-gaps, plus + // the children that fall out of the rule-3 tightening). At least + // one ::warning:: annotation must appear. + let warning_count = stdout.matches("::warning").count(); + assert!( + warning_count >= 1, + "expected at least one ::warning:: annotation in --warn-only mode; got:\n{stdout}" + ); + + // Each annotation must have the documented shape. + for line in stdout.lines().filter(|l| l.contains("::warning")) { + assert!( + line.starts_with("::warning file="), + "annotation must use `file=` payload; got: {line}" + ); + assert!( + line.contains("rivet docs check --coverage"), + "annotation must reference the gate; got: {line}" + ); + } +} + +/// `--warn-only` and `--strict` are mutually exclusive (clap-enforced): +/// invoking both must fail with a clap error before any work runs. +#[test] +fn coverage_warn_only_and_strict_are_mutually_exclusive() { + let output = Command::new(rivet_bin()) + .args(["docs", "check", "--coverage", "--warn-only", "--strict"]) + .output() + .expect("failed to execute rivet with both flags"); + + assert!( + !output.status.success(), + "clap must reject --warn-only + --strict together" + ); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + stderr.contains("cannot be used with") || stderr.contains("conflicts"), + "expected clap conflict error; got stderr:\n{stderr}" + ); } /// `--strict` exits non-zero whenever the inventory has any uncovered -/// path. With the current TOPICS registry we know there are at least a -/// few uncovered commands (variant, baseline, snapshot, runs, pipelines, -/// templates, close-gaps), so strict mode must currently fail. Once -/// those gaps are filled the test still holds: if NOTHING is uncovered, -/// strict exits 0, but then `expected_uncovered_count >= 1` is the only -/// place we assert non-zero — re-flip when the world catches up. +/// path. With the current TOPICS registry we know there are uncovered +/// commands (variant, baseline, snapshot, runs, pipelines, templates, +/// close-gaps, plus the rule-3-tightening children), so strict mode +/// must currently fail. The test still holds the day the inventory +/// becomes complete: it asserts the (status,output) pair is internally +/// consistent. #[test] fn coverage_strict_fails_when_uncovered_present() { let output = Command::new(rivet_bin()) @@ -114,6 +182,24 @@ fn coverage_strict_fails_when_uncovered_present() { } } +/// On the current main branch, the repo's TOPICS registry leaves +/// several subcommands uncovered (with #248's tightened rule 3 the +/// count is even larger than before). Pin that as a regression check — +/// `--coverage --strict` must currently exit 1. +#[test] +fn coverage_strict_currently_fails_on_main() { + let output = Command::new(rivet_bin()) + .args(["docs", "check", "--coverage", "--strict"]) + .output() + .expect("failed to execute rivet docs check --coverage --strict"); + + assert!( + !output.status.success(), + "strict mode is expected to fail on current main (uncovered subcommands present); \ + if this passes, the inventory is now clean and the test should be flipped." + ); +} + /// JSON output is machine-readable and follows the standard envelope /// (`command`, `status`, `total`, `covered`, `uncovered`, `subcommands`). #[test]