feat(0.2): truth-verify gate — feature-status doc ⊆ signal manifest (Track 9.7)#156
Open
feat(0.2): truth-verify gate — feature-status doc ⊆ signal manifest (Track 9.7)#156
Conversation
…Track 9.7)
Adds the Track 9.7 release-readiness gate: cross-checks
docs/release/feature-status.md against the canonical signal
manifest. Drift between what the curated doc promises and what the
engine actually ships is the failure mode adopters notice when they
evaluate the binary against the marketing claim.
The gate
- New `cmd/terrain-truth-verify/main.go` walks
docs/release/feature-status.md, extracts every backtick-delimited
camelCase signal reference between the "Detectors / signal
types" anchor and the "Planned" subsection, and verifies each
resolves to a real entry in internal/signals/manifest.go.
- Camel-case constraint (lowercase start + at least one mid-word
uppercase) excludes CLI verbs (`report`, `eval`, `policy`) and
English filler words from false-positive matches.
- Planned subsection excluded by design — references there name
signals that *don't* yet have a code-side implementation;
flagging them would invert the signal.
- Engine self-diagnostic signals (detectorPanic,
detectorBudgetExceeded, detectorMissingInput, suppressionExpired)
excluded from the orphan check — they're documented inline
alongside the mechanisms that emit them, not in the curated
signal table.
Wired into Makefile as `make truth-verify`. Same posture as
`make docs-verify` and `make pillar-parity` — fails with
`::error::N broken signal reference(s)` + per-signal output
suitable for CI annotation.
What it caught
Initial run on the current docs surfaced one real broken
reference: `duplicateCluster` was listed as a "stable signal" in
the table but isn't a `Signal` type at all — duplicate-cluster
analysis surfaces via `DuplicateClusters` in the analyze report
rather than through the signal mechanism. Doc updated to clarify
the distinction.
Plus 15 advisory orphans (stable signals in the manifest that the
curated doc doesn't mention by name). The doc explicitly says it's
a "curated view" with the manifest as the source of truth, so
these are acceptable today; --strict-orphans escalates them to
hard failures for adopters who want full coverage.
Out of scope today (per the package comment)
- README command list ⊆ dispatcher: needs Track 9.6 registry
refactor first
- CHANGELOG promotion-claim cross-check: per-signal status is
already manifest-driven, so docs-verify catches it
- CI matrix ⊆ compatibility tier doc: distinct failure mode in
workflow YAML rather than markdown
Coverage
5 new unit tests in main_test.go: happy-path extraction, no-anchor
returns empty, planned subsection excluded, all-lowercase tokens
rejected (regression guard against the false-positive class that
caught report / eval / policy on the first run), engine-diagnostic
classifier.
Verification: full Go test suite green; make truth-verify exits 0
on the current docs surface; make docs-verify clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
[INFO] Terrain — Informational only
Coverage gaps in changed code
Recommended tests1 test(s) with exact coverage of 0 impacted unit(s). 1 impacted unit(s) have no covering tests in the selected set.
Limitations
Generated by Terrain · Targeted Test ResultsTerrain selected 1 test(s) instead of the full suite.
|
Terrain AI Risk Review
Decision: PASS — AI surfaces are covered. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
make truth-verify— the Track 9.7 release-readiness gatethat cross-checks
docs/release/feature-status.mdagainst thecanonical signal manifest. Drift between the curated doc and the
shipping engine is the failure mode adopters notice when they
evaluate the binary against the marketing claim.
cmd/terrain-truth-verify/main.gowalks the doc, extractsevery backtick-delimited camelCase signal reference between the
detector-section and Planned-section anchors, verifies each
resolves to a real manifest entry
make truth-verifyMakefile targetduplicateClusterwaslisted as a stable signal but isn't a
Signaltype — surfacesvia
DuplicateClustersin the analyze report instead)What changed
New code:
cmd/terrain-truth-verify/main.go— verifier (~180 lines, dependsonly on internal/signals + internal/models)
cmd/terrain-truth-verify/main_test.go— 5 unit testsNew Makefile target:
truth-verifyFixed doc drift:
docs/release/feature-status.md—duplicateClusterrow reframed to clarify it's surfaced viaDuplicateClustersnot via the signal mechanismDrift posture
entry. Real broken promise.
surfaced in the curated doc. Acceptable today since the doc is
explicitly a "curated view" with the manifest as the source of
truth. Pass
--strict-orphansto escalate.Out of scope (documented in the package comment)
refactor first
already manifest-driven, so
make docs-verifycatches itworkflow YAML rather than markdown
Test plan
go build ./...cleango test ./cmd/terrain-truth-verify/...— 5 new tests(happy path, no-anchor, planned-subsection exclusion,
all-lowercase-token rejection regression guard, engine-
diagnostic classifier)
make truth-verifyexits 0 on current docsmake docs-verifycleanPlan tracker
Closes Track 9.7. Track 9 remaining: 9.2 (panic recovery
completion), 9.5 (pipeline architectural separation), 9.6
(registry refactor). All explicitly post-0.2.0-blocking.
🤖 Generated with Claude Code