feat: E2E observability + eval infrastructure + all skills templated#55

Merged

garrytan merged 33 commits intomainfrom

v0.3.6-qa-upgrades

Mar 14, 2026

Commits on Mar 14, 2026

chore: regenerate SKILL.md from template
garrytan
committed
feat: add remote slug helper and auto-gitignore for .gstack/
garrytan
committed
feat: QA test plan tiers with per-page risk scoring
garrytan
committed
feat: dual greptile-history paths (per-project + global)

garrytan
and
claude
committed
Merge remote-tracking branch 'origin/main' into v0.3.5-qa-upgrades
garrytan
committed
feat: 3-tier eval suite with planted-bug outcome testing (EVALS=1)

garrytan
and
claude
committed
fix: pass all LLM evals — severity defs, rubric edge cases, EVALS=1 flag

garrytan
and
claude
committed
simplify: one command for evals — bun run test:evals

garrytan
and
claude
committed
fix: rewrite session-runner to claude -p subprocess, lower flaky baselines

garrytan
and
claude
committed
Merge remote-tracking branch 'origin/main' into v0.3.6-qa-upgrades
garrytan
committed
feat: stream-json NDJSON parser for real-time E2E progress

garrytan
and
claude
committed
feat: eval persistence with auto-compare against previous run

garrytan
and
claude
committed
feat: eval CLI tools + docs cleanup

garrytan
and
claude
committed
fix: update check preamble exits 1 when up to date — convert all skills to .tmpl

garrytan
and
claude
committed
fix: remove false-positive Exit code 1 pattern, fix NEEDS_SETUP test, update QA tests

garrytan
and
claude
committed
fix: lower planted-bug detection baselines and LLM judge thresholds for reliability

garrytan
and
claude
committed
fix: make planted-bug evals resilient to max_turns and browse error flakes

garrytan
and
claude
committed
fix: simplify planted-bug eval prompts for reliable 25-turn completion

garrytan
and
claude
committed
fix: 100% E2E pass — isolate test dirs, restart server, relax FP thresholds

garrytan
and
claude
committed
test: add update-check exit code regression tests

garrytan
and
claude
committed
feat: template-ify all skills + E2E tests for plan-ceo-review, plan-eng-review, retro

garrytan
and
claude
committed
fix: increase timeouts for plan-review and retro E2E tests

garrytan
and
claude
committed
fix: plan-ceo-review timeout — init git repo, skip codebase exploration, bump to 420s

garrytan
and
claude
committed
feat: add E2E observability — heartbeat, progress.log, NDJSON persistence, savePartial()

garrytan
and
claude
committed
feat: wire runId + testName + diagnostics through all E2E tests

garrytan
and
claude
committed
feat: eval-watch dashboard + observability unit tests (15 tests, 11 codepaths)

garrytan
and
claude
committed
fix: detect is_error from claude -p result line (ConnectionRefused was PASS)

garrytan
and
claude
committed
fix: never clean up observability artifacts — partial file persists after finalize

garrytan
and
claude
committed
fix: fail fast on API connectivity — pre-check before E2E suite

garrytan
and
claude
committed
chore: bump version and changelog (v0.3.6)

garrytan
and
claude
committed
docs: update README, CONTRIBUTING, ARCHITECTURE for v0.3.6

garrytan
and
claude
committed
fix: auto-clear stale heartbeat when process is dead

garrytan
and
claude
committed
fix: update check ignores stale UP_TO_DATE cache after version change

garrytan
and
claude
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: E2E observability + eval infrastructure + all skills templated#55

feat: E2E observability + eval infrastructure + all skills templated#55
garrytan merged 33 commits intomainfrom
v0.3.6-qa-upgrades

Commits on Mar 14, 2026

chore: regenerate SKILL.md from template

feat: add remote slug helper and auto-gitignore for .gstack/

feat: QA test plan tiers with per-page risk scoring

feat: dual greptile-history paths (per-project + global)

Merge remote-tracking branch 'origin/main' into v0.3.5-qa-upgrades

feat: 3-tier eval suite with planted-bug outcome testing (EVALS=1)

fix: pass all LLM evals — severity defs, rubric edge cases, EVALS=1 flag

simplify: one command for evals — bun run test:evals

fix: rewrite session-runner to claude -p subprocess, lower flaky baselines

Merge remote-tracking branch 'origin/main' into v0.3.6-qa-upgrades

feat: stream-json NDJSON parser for real-time E2E progress

feat: eval persistence with auto-compare against previous run

feat: eval CLI tools + docs cleanup

fix: update check preamble exits 1 when up to date — convert all skills to .tmpl

fix: remove false-positive Exit code 1 pattern, fix NEEDS_SETUP test, update QA tests

fix: lower planted-bug detection baselines and LLM judge thresholds for reliability

fix: make planted-bug evals resilient to max_turns and browse error flakes

fix: simplify planted-bug eval prompts for reliable 25-turn completion

fix: 100% E2E pass — isolate test dirs, restart server, relax FP thresholds

test: add update-check exit code regression tests

feat: template-ify all skills + E2E tests for plan-ceo-review, plan-eng-review, retro

fix: increase timeouts for plan-review and retro E2E tests

fix: plan-ceo-review timeout — init git repo, skip codebase exploration, bump to 420s

feat: add E2E observability — heartbeat, progress.log, NDJSON persistence, savePartial()

feat: wire runId + testName + diagnostics through all E2E tests

feat: eval-watch dashboard + observability unit tests (15 tests, 11 codepaths)

fix: detect is_error from claude -p result line (ConnectionRefused was PASS)

fix: never clean up observability artifacts — partial file persists after finalize

fix: fail fast on API connectivity — pre-check before E2E suite

chore: bump version and changelog (v0.3.6)

docs: update README, CONTRIBUTING, ARCHITECTURE for v0.3.6

fix: auto-clear stale heartbeat when process is dead

fix: update check ignores stale UP_TO_DATE cache after version change