Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion cmd/terrain/cmd_analyze.go
Original file line number Diff line number Diff line change
Expand Up @@ -112,10 +112,22 @@ func runInit(root string, jsonOutput bool) error {
fmt.Println()
}
if result.PolicyPath != "" {
fmt.Printf(" %d. Edit .terrain/policy.yaml to enable governance rules\n", step)
fmt.Printf(" %d. Edit .terrain/policy.yaml — three starter policies live\n", step)
fmt.Println(" under docs/policy/examples/{minimal,balanced,strict}.yaml")
step++
fmt.Println()
}

// CI integration pointer — Track 8.4. Always shown so adopters
// see the whole ladder from `terrain init` onwards. The trust-
// ladder doc explains the four-rung adoption path; the example
// workflow is the one canonical CI config.
fmt.Printf(" %d. Wire Terrain into CI (warn-only by default):\n", step)
fmt.Println(" Copy docs/examples/gate/github-action.yml to .github/workflows/")
fmt.Println(" The trust ladder (docs/product/trust-ladder.md) explains")
fmt.Println(" when to flip on blocking gates.")
fmt.Println()

return nil
}

Expand Down
109 changes: 109 additions & 0 deletions docs/examples/gate/github-action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Terrain — recommended GitHub Action workflow.
#
# This is the ONE config the project recommends. Drop it into
# `.github/workflows/terrain-pr.yml`, commit, and you have:
#
# - per-PR `terrain analyze` running on every push
# - per-PR `terrain report pr` posting a unified comment with the
# test-system risk for the diff
# - safe-default mode by default (warn, don't block) so day-one
# adoption doesn't brick CI on existing repo debt
# - a one-line opt-in to flip on blocking gates once the warning
# output is calibrated for your repo
#
# The two-step flow at the heart of Terrain (`terrain analyze` →
# `terrain report pr`) is the workflow's spine. Everything else is
# an optional add-on.
#
# Reference: https://github.com/pmclSF/terrain
# Vision: docs/product/vision.md
# Trust: docs/product/trust-ladder.md (which mode to run when)
# Policy: docs/policy/examples/{minimal,balanced,strict}.yaml

name: Terrain

on:
pull_request:
branches: [main]
workflow_dispatch:

# Cancel an older Terrain run when a newer commit lands on the same PR.
concurrency:
group: terrain-${{ github.ref }}
cancel-in-progress: true

permissions:
contents: read
pull-requests: write # required for the PR-comment step
checks: write # required for SARIF / annotations

jobs:
terrain:
name: Terrain
runs-on: ubuntu-latest
timeout-minutes: 15

steps:
- name: Check out the PR head
uses: actions/checkout@v4
with:
fetch-depth: 0 # full history so `--base main` diffs work

- name: Install Terrain
# Pick the install path that matches your runner image.
# Homebrew is the lowest-friction default; npm needs cosign;
# `go install` works on any image with Go 1.23+.
run: |
brew install pmclSF/terrain/mapterrain
terrain version

# ── Step 1 of the primary workflow: understand the test system ──
- name: terrain analyze
run: |
terrain analyze \
--write-snapshot \
--json > terrain-snapshot.json

# ── Step 2: gate the PR ────────────────────────────────────────
# Default: warn-only. The job stays green; the PR comment shows
# what Terrain found. Established repos with debt won't brick CI
# on day one.
#
# To enable blocking gates, uncomment the `--fail-on critical`
# line below. Combined with `--new-findings-only --baseline ...`,
# the gate fires only on findings introduced AFTER the snapshot
# in the baseline file (see `terrain analyze --write-snapshot`
# for how to persist the baseline).
- name: terrain report pr (warn-only by default)
run: |
terrain report pr \
--base ${{ github.event.pull_request.base.sha }} \
--format markdown \
--new-findings-only \
--baseline terrain-snapshot.json \
> terrain-pr-comment.md
# --fail-on critical # uncomment to gate the build

- name: Post or update the PR comment
uses: peter-evans/create-or-update-comment@v5
if: github.event_name == 'pull_request'
with:
issue-number: ${{ github.event.pull_request.number }}
body-path: terrain-pr-comment.md
edit-mode: replace
# Stable comment marker so successive runs update the same
# thread instead of stacking new comments.
body-includes: '<!-- terrain-pr-report -->'

# Optional: upload SARIF for GitHub code scanning. Adds Terrain
# findings to the Security tab alongside CodeQL / Snyk / etc.
- name: terrain analyze (SARIF)
run: |
terrain analyze --format sarif --redact-paths > terrain.sarif

- name: Upload SARIF to code scanning
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: terrain.sarif
category: terrain
141 changes: 141 additions & 0 deletions docs/product/trust-ladder.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# Terrain Trust Ladder

How adopters move from "see what Terrain finds" to "block PRs on
what Terrain finds." Four rungs, each a clear next step. Don't skip
rungs — every team that does ends up with a noisy gate they can't
defend.

## Why a ladder

Terrain reports findings against the *current state* of your repo.
On day one, that state usually includes inherited debt: untested
exports a previous team shipped, AI surfaces without eval coverage,
mocks that crept in over years. A blocking gate against that
backlog brick CI on the first PR a contributor opens.

The ladder solves this by separating **visibility** from **gating**.
Each rung gives you more information; only the upper rungs block
merges.

## Rung 1 — Inventory

**What you do:** install Terrain locally, run `terrain analyze`, read
the report.

**What you get:** the test universe mapped — frameworks, test files,
code units, AI surfaces, eval scenarios, ownership, coverage gaps.
A baseline understanding of where your test system stands.

**What it doesn't do:** affect your CI. Run this rung locally for as
long as you want before promoting.

```bash
brew install pmclSF/terrain/mapterrain
cd your-repo
terrain analyze
```

**Move up when:** the report makes sense and you can describe
what it shows to a colleague.

## Rung 2 — Warnings

**What you do:** add the [recommended GitHub Action](../examples/gate/github-action.yml)
to your repo. Default config is warn-only.

**What you get:** every PR gets a Terrain comment with the
change-scoped risk report. Findings are visible in the PR review
flow; the build stays green.

**What it doesn't do:** block any merge. Surface things; let humans
decide.

```yaml
# In .github/workflows/terrain-pr.yml — copy from
# docs/examples/gate/github-action.yml
# (no --fail-on flag; warn-only is the default)
```

**Move up when:** the comments have run for two-to-four weeks and
the warning surface is calibrated — false positives are filed,
suppressions are in place, and the team agrees on which findings
should block.

## Rung 3 — CI annotations

**What you do:** add SARIF upload to the workflow (already in the
recommended template) so findings flow into the Security tab.
Optionally add `--format annotation` so PR-level annotations appear
in the diff view.

**What you get:** findings live in two places: the PR comment (high
signal, prose-shaped) and the Security tab (each finding addressable
by URL, navigable per-line in the diff). Reviewers can comment on a
specific Terrain finding the same way they comment on a CodeQL one.

**What it doesn't do:** block merges. SARIF surface is for review,
not enforcement.

**Move up when:** reviewers are routinely engaging with Terrain
findings in PRs and the team is ready to put a stake in the ground:
"from now on, no new HIGH-severity finding ships."

## Rung 4 — Blocking gates

**What you do:** flip on `--fail-on <severity>` in the workflow.
Pair with `--new-findings-only --baseline <path>` so existing debt
is grandfathered in.

**What you get:** the gate the platform team has been planning
for. CI fails on net-new findings at or above the chosen severity.
Suppressions remain the escape valve for legitimate waivers, with
required reasons + optional expiry.

```yaml
# Uncomment this line in the recommended template:
# --fail-on critical
```

**Recommended pairing per pillar:**

- Severity gate: `--fail-on critical` (start) → `--fail-on high`
(mature) → `--fail-on medium` (zero-tolerance branches)
- Baseline: `--new-findings-only --baseline
.terrain/snapshots/latest.json`
- Policy: copy [`docs/policy/examples/balanced.yaml`](../policy/examples/balanced.yaml)
to start; promote to [`strict.yaml`](../policy/examples/strict.yaml) over time
- Suppressions: `terrain suppress <finding-id> --reason "..." --expires
YYYY-MM-DD --owner @platform` for legitimate waivers

**This rung is the destination.** Most teams settle here.

## What's NOT on the ladder

- **Hand-graded PR reviews of every Terrain finding.** Doesn't
scale; the gate exists so reviewers can spend their time
elsewhere.
- **Custom-thresholded per-team policy variants.** 0.2 ships
one repo-wide policy. Per-team variants are tracked for 0.3.
- **Auto-fix of detected findings.** Terrain reports; humans fix.
AST-grade auto-fix is on the long-term plan but explicitly not
in 0.2 or 0.3.

## Common adoption mistakes

| Mistake | Why it fails | Fix |
|---------|--------------|-----|
| Jumping from Rung 1 to Rung 4 | Existing debt brick CI on day one | Pair with `--new-findings-only --baseline ...` always |
| Suppressions without `expires` | Waivers accumulate, audit becomes impossible | Default `--expires` to 90-180 days; renew or remove |
| Ignoring the AI Risk Review section | Heuristic detectors fire false positives in 0.2 — but signal-to-noise is good enough that ignoring is a real loss | Triage with `terrain explain finding <id>`; suppress or fix |
| Treating Tier-2 capabilities as Tier-1 | Marketing gets ahead of evidence; review scrutiny exposes it | Read `docs/release/feature-status.md` before claiming things publicly |

## Next reading

- [`docs/product/vision.md`](vision.md) — the product story behind
the ladder
- [`docs/release/feature-status.md`](../release/feature-status.md) —
per-capability tier so you know what's safe to lean on
- [`docs/policy/examples/`](../policy/examples/) — three starter
policies matched to the ladder rungs
- [`docs/examples/gate/github-action.yml`](../examples/gate/github-action.yml) —
the one recommended CI config
Loading