Search results

CI pipeline: run mock scenarios and verify bundles

Status: Open.

#15 In cmangun/agentic-eval-harness;

· cmangun opened on Feb 21, 2026

Add golden bundles for mock runs

Status: Open.

#14 In cmangun/agentic-eval-harness;

· cmangun opened on Feb 21, 2026

Implement evidence export generator (JSON + HTML)

Status: Open.

#13 In cmangun/agentic-eval-harness;

· cmangun opened on Feb 21, 2026

Scenario 08: Artifact production (manifest + verification)

Status: Open.

#11 In cmangun/agentic-eval-harness;

· cmangun opened on Feb 21, 2026

Implement metrics aggregator (latency/errors/tool calls)

Status: Open.

#12 In cmangun/agentic-eval-harness;

· cmangun opened on Feb 21, 2026

Scenario 07: Deterministic run (stable trace hash)

Status: Open.

#10 In cmangun/agentic-eval-harness;

· cmangun opened on Feb 21, 2026

Scenario 06: Policy bypass attempt (must deny + record)

Status: Open.

#9 In cmangun/agentic-eval-harness;

· cmangun opened on Feb 21, 2026

Scenario 05: Tool failure recovery (retry + evidence)

Status: Open.

#8 In cmangun/agentic-eval-harness;

· cmangun opened on Feb 21, 2026

Scenario 03: Budget cap (stop/ask approval when exceeded)

Status: Open.

#6 In cmangun/agentic-eval-harness;

· cmangun opened on Feb 21, 2026

Scenario 04: Human approval gate (pause and request approval)

Status: Open.

#7 In cmangun/agentic-eval-harness;

· cmangun opened on Feb 21, 2026

Scenario 01: Retrieval under policy (PHI redaction required)

Status: Open.

#4 In cmangun/agentic-eval-harness;

· cmangun opened on Feb 21, 2026

Scenario 02: Tool schema enforcement (invalid args must fail)

Status: Open.

#5 In cmangun/agentic-eval-harness;

· cmangun opened on Feb 21, 2026