Pipeline Failure Root Cause Auto-Analyzer with Pattern Extraction

[github-actions]

Strategic Improvement

Currently at 75% success rate vs 85% target. None of the open issues systematically analyze WHY pipelines are failing. Build an analyzer that processes the last N failed pipelines, extracts error patterns, classifies by type (context exhaustion, test failure, merge conflict, etc.), ranks by frequency, and outputs an actionable report with recommended fixes. This diagnostic engine feeds all the retry/recovery improvements already in flight.

Acceptance Criteria

• Analyze events.jsonl and pipeline-artifacts for last 50 failed pipeline runs
• Extract and classify failure patterns by type with frequency counts
• Generate ranked report of top-5 failure modes with specific examples
• Output machine-readable JSON with failure signatures for use by other tools
• Include recommended mitigations for each failure pattern
• Run automatically after every 10 pipeline failures to keep analysis fresh

Context

• Priority: P0
• Complexity: standard
• Generated by: Strategic Intelligence Agent
• Strategy alignment: P0: Reliability & Success Rate

[github-actions]

⚓ Shipwright Pipeline Starting

Field	Value
Template	standard
Complexity	standard
Max Iterations	20
Triage	P0 strategic reliability feature—clear acceptance criteria, moderate scope (~1200 LOC), directly addresses 75%→85% success rate target.
Runner	ubuntu-latest
Risk Level	unknown
Workflow	24282387870

The autonomous pipeline is processing this issue. Updates will be posted as stages complete.

[github-actions]

[github-actions]

[github-actions]

📋 Implementation Plan

Click to expand full plan (15 lines)

The implementation plan is written to .claude/plan.md. Here's a summary of the key architectural decisions:

Architecture: New standalone CLI script scripts/sw-failure-analysis.sh (accessible via shipwright failure-analysis or shipwright fa) that sources the existing lib/root-cause.sh classification engine. This keeps the single-error classifier separate from the aggregate analyzer.

Data sources: Reads from 4 existing JSONL files — outcomes.jsonl (primary, one line per pipeline run), events.jsonl (stage-level correlation), root-causes.jsonl (pre-classified errors), and error-log.jsonl (raw errors).

Key deliverables:

• 6 core functions: collect, correlate, classify, extract signatures, generate report, generate JSON
• Machine-readable JSON output at ~/.shipwright/optimization/failure-analysis.json
• Auto-trigger every 10 failures via atomic counter in rootcause_learn()
• Comprehensive test suite (~25 tests)

Files touched: 2 new (script + tests), 4 modified (CLI router, package.json, event schema, root-cause.sh auto-trigger hook).

Risk mitigations: flock for counter concurrency, tail -N for large file performance, graceful handling of missing/malformed data, full rollback is a simple delete + 3-line revert.

---

Generated by shipwright pipeline at 2026-04-11T12:34:31Z

[github-actions]

[github-actions]

[github-actions]

📐 Architecture Decision Record

Click to expand ADR (19 lines)

The ADR has been written to docs/superpowers/specs/2026-04-11-failure-analysis-design.md. Here's a summary of the key architectural decisions:

Chosen approach: New standalone scripts/sw-failure-analysis.sh CLI script that sources the existing lib/root-cause.sh classification engine. This cleanly separates individual error classification (existing) from aggregate pattern analysis (new).

Key design properties:

• Read-only ETL pipeline — never mutates source data, naturally idempotent
• 4 JSONL data sources — outcomes.jsonl (primary), events.jsonl (correlation), root-causes.jsonl (pre-classified), error-log.jsonl (raw)
• 5-stage processing — collect, correlate, classify, extract signatures, output
• Auto-trigger — atomic counter in rootcause_learn() fires analysis every 10 failures, guarded by flock
• Graceful degradation — every component handles its own errors, returns safe defaults, never blocks the pipeline

Rejected alternatives:

1. Extending lib/root-cause.sh — violates single responsibility (427-line library would double in size)
2. Building into sw-memory.sh — couples analytics into 2240-line storage layer
3. Dashboard-only (TypeScript) — primary consumers are bash-based daemon/memory systems

Blast radius: 2 new files, 4 modified files (1-line changes each). Rollback is delete + 4 one-line reverts. No data migration needed.

---

Generated by shipwright pipeline design stage at 2026-04-11T12:44:40Z

[github-actions]

[github-actions]

[github-actions]

🔨 Build started — shipwright loop with 20 max iterations, 1 agent(s), model: opus

[github-actions]

❌ Pipeline failed at stage build after 9m 17s.

No log available