Skip to content

Track canonical-emitted-IDs test is hand-maintained (deliberate trade-off) #67

@Zious11

Description

@Zious11

Background

PR #66 adds `tests/mitre_tests.rs::known_emitted_technique_ids_resolve_in_lookup` — a hand-curated list of every `mitre_technique: Some("...")` value the codebase emits, asserted to resolve via `technique_name` and `technique_tactic`. Both local PR review (silent-failure-hunter) and Copilot review flagged that this test is not mechanically exhaustive: a new emission site can be added in an analyzer without updating the list, and CI will not fail.

Why deliberate

External validation (Perplexity) explicitly recommended the hand-curated test as the idiomatic Rust pattern at this scale (~6 emission sites). Specifically:

  • A `build.rs` regex scan of `src/` for the literal pattern `mitre_technique:\s*Some\("([^"]+)"` is fragile (misses `.into()`, `format!(...)`, refactors).
  • A `proc-macro` requiring every emission to use a registration macro is invasive.
  • `inventory` / `linkme` add a dependency for marginal benefit at small scale.
  • A runtime registration via `ctor` adds startup overhead and undefined init order across modules.

The hand-curated test is the cheapest pattern that still catches the most common failure mode: typo'd technique IDs (`Some("T10046")` instead of `Some("T1046")`).

Trigger condition

Revisit this if emission sites grow > ~20, OR if a future contributor introduces a typo'd ID that ships to a release because the list wasn't updated. At that point, evaluate:

  1. Static lookup table (`&'static [(&str, &str, MitreTactic)]`) replacing the parallel `match` arms entirely; tests iterate the table directly.
  2. Build-script that emits a generated test from a manifest file (less fragile than regex of `src/`).
  3. Switch to `linkme` / `inventory` once the dependency is justified by other use cases too.

Acceptance criteria for closing this issue

EITHER:

  • A documented trigger event has occurred (>20 emission sites, or a missed-update incident).
  • AND a chosen approach (1, 2, or 3 above) is implemented with a CI-enforced cross-check.

OR:

  • The trade-off remains acceptable; this issue serves as the tracked rationale.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationtestTest coverage

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions