feat(action-brain): Action Brain MVP 0.1b — wacli collector, auto-ingest pipeline, checkpoint health (v0.10.2)

CHANGELOG.md

@@ -2,6 +2,32 @@
 
 All notable changes to GBrain will be documented in this file.
 
+## [0.10.2] - 2026-04-17
+
+### Added
+
+- **`gbrain action run` — cron-ready auto-ingest pipeline.** One command reads new WhatsApp messages from the wacli store, extracts commitments with the LLM extractor, and stores them in Action Brain. Checkpoint-aware: skips already-processed messages. Staleness gate: bails out if wacli data is older than `--stale-after-hours` (default 24h). Returns structured JSON with counts and errors — CI/cron-friendly.
+- **Morning brief is now checkpoint-aware.** `gbrain action brief` auto-reads the wacli checkpoint to compute message freshness — no more manually passing `--last-sync-at`. Pass `--checkpoint-path` to override the default location.
+- **Action item creation now returns idempotency signal.** `createItemWithResult()` tells callers whether an item was freshly inserted or already existed — so the ingest pipeline can report accurate created/skipped counts without extra DB queries.
+- **Wacli health checks before every ingest run.** The auto-ingest runner now verifies wacli's health state (`healthy` / `degraded` / `failed`) before touching your messages. A stale store (>24h no update) is reported as `degraded`; a disconnected store is `failed`. Scheduled runs can fail fast on degraded health with `--fail-on-degraded`, so your cron doesn't silently ingest stale data.
+- **Extraction retries on transient LLM errors.** `extractCommitments()` now retries on overload, rate limit, and timeout errors (configurable, default 1 retry) so a momentary API hiccup doesn't drop commitments from your pipeline.
+- **Commitment actor normalization.** `stabilizeCommitments()` grounds LLM output against the actual message context — if the LLM assigns a commitment to the wrong person, the pipeline corrects it using message-level "X will..." pattern matching. Fewer ghost obligations attributed to the wrong contact.
+- **`gbrain action run` exits non-zero when ingest fails.** Schedulers and cron jobs can now reliably detect degraded or unhealthy runs by exit code — no JSON parsing required. A healthy run exits 0 and still emits the full structured summary. A failed/degraded run exits 1, also with full JSON output, so you get both machine-readable failure detection and human-readable diagnosis.
+
+### Fixed
+
+- **Replay now produces stable, deduplicated source IDs.** When the same WhatsApp message is ingested twice — even if the LLM extracts slightly different wording or commitment type — the second run sees the existing item and skips it cleanly. Source IDs are now ordinal-based per message (`msg-id:ab:0`, `msg-id:ab:1`) rather than content-hashed, so dedup is reliable regardless of LLM drift between runs.
+- **`action_ingest` reports accurate created/skipped counts.** The operation now correctly reports how many items were freshly created vs. already existed, using the idempotency signal from the storage layer.
+- **Brief freshness is now always current.** The wacli checkpoint now records a heartbeat timestamp on every successful poll — even when no new messages arrive. This means `gbrain action brief` no longer shows a stale freshness warning when wacli is healthy but simply has no new traffic.
+- **Source IDs are now isolated per wacli store.** Two stores that happen to share the same raw message ID can no longer collide in the dedup index — each store gets its own namespace. Prevents cross-store duplicate suppression when you add a second wacli source.
+- **Ambiguous bare source IDs now fail closed.** When a commitment's source ID can't be unambiguously resolved to one message (e.g., a bare ID that exists in multiple stores), the extractor rejects the batch instead of silently attributing it to the wrong message. Prevents ghost commitments from being stored with incorrect provenance.
+
+## [0.10.1] - 2026-04-16
+
+### Added
+
+- **Wacli collector with checkpoint store.** `collector.ts` reads your WhatsApp export files from the wacli local store and maintains a checkpoint so the ingest pipeline only processes new messages since the last run. Deterministic parsing, dedup-safe, and cron-friendly — no duplicate ingestion on re-runs.
+
 ## [0.10.0] - 2026-04-16
 
 ### Added

CLAUDE.md

@@ -63,10 +63,12 @@ markdown files (tool-agnostic, work with both CLI and plugin contexts).
 - `openclaw.plugin.json` — ClawHub bundle plugin manifest
 - `src/action-brain/types.ts` — Action Brain shared types (ActionItem, CommitmentBatch, ExtractionResult)
 - `src/action-brain/action-schema.ts` — PGLite DDL + idempotent schema init for action_items / action_history tables
-- `src/action-brain/action-engine.ts` — Storage layer: CRUD, priority scoring (urgency × confidence × recency), PGLite lifecycle
-- `src/action-brain/extractor.ts` — LLM commitment extraction (two-tier Haiku→Sonnet), XML delimiter defense, stable source IDs
-- `src/action-brain/brief.ts` — Morning priority brief generator: ranked action items, overdue detection, deduplication
-- `src/action-brain/operations.ts` — 5 Action Brain operations (action_list, action_brief, action_resolve, action_mark_fp, action_ingest)
+- `src/action-brain/action-engine.ts` — Storage layer: CRUD, priority scoring (urgency × confidence × recency), PGLite lifecycle; `createItemWithResult()` returns idempotency signal (created vs skipped)
+- `src/action-brain/extractor.ts` — LLM commitment extraction (Sonnet default; quality gate uses Haiku→Sonnet escalation), `stabilizeCommitments()` for message-grounded actor normalization, XML delimiter defense, stable source IDs, owner context injection
+- `src/action-brain/brief.ts` — Morning priority brief generator: ranked action items, overdue detection, deduplication; freshness reads from wacli checkpoint (not action item creation time)
+- `src/action-brain/collector.ts` — Wacli message collector: invokes `wacli messages list` CLI, deduplicates by message ID, checkpoint-aware cursor (skips already-processed messages), heartbeat freshness on no-op polls
+- `src/action-brain/ingest-runner.ts` — Auto-ingest orchestrator: preflight checks, staleness gate, collect → extract → store pipeline; cron-ready, returns structured JSON
+- `src/action-brain/operations.ts` — 6 Action Brain operations (action_list, action_brief, action_resolve, action_mark_fp, action_ingest, action_ingest_auto)
 
 ## Commands
 
@@ -78,7 +80,7 @@ Key commands added in v0.7:
 
 ## Testing
 
-`bun test` runs all tests (33 unit test files + 5 E2E test files). Unit tests run
+`bun test` runs all tests (44 unit test files + 6 E2E test files). Unit tests run
 without a database. E2E tests skip gracefully when `DATABASE_URL` is not set.
 
 Unit tests: `test/markdown.test.ts` (frontmatter parsing), `test/chunkers/recursive.test.ts`
@@ -104,13 +106,23 @@ parity), `test/cli.test.ts` (CLI structure), `test/config.test.ts` (config redac
 `test/eval.test.ts` (retrieval metrics: precisionAtK, recallAtK, mrr, ndcgAtK, parseQrels),
 `test/action-brain/action-schema.test.ts` (Action Brain DDL + idempotent init),
 `test/action-brain/action-engine.test.ts` (CRUD, scoring, PGLite lifecycle),
-`test/action-brain/extractor.test.ts` (extraction, source ID stability, injection defense, timestamp bounds),
+`test/action-brain/extractor.test.ts` (extraction, stabilizeCommitments actor normalization, source ID stability, injection defense, timestamp bounds, owner context),
 `test/action-brain/brief.test.ts` (brief generation, scoring, dedup, overdue detection),
-`test/action-brain/operations.test.ts` (all 5 ops, ingest trust boundary, batch fallbacks).
+`test/action-brain/collector.test.ts` (wacli file reading, checkpoint store, dedup, freshness filtering),
+`test/action-brain/ingest-runner.test.ts` (preflight checks, staleness gate, collect/extract/store pipeline, structured JSON output),
+`test/action-brain/operations.test.ts` (all 6 ops, ingest trust boundary, batch fallbacks, action_ingest_auto pipeline),
+`test/action-brain/e2e-live-validation-metrics.test.ts` (matchCommitment unit tests: alias matching, type compatibility, action substring matching),
+`test/embed.test.ts` (embedding batch + retry logic), `test/import-walker.test.ts` (file walker + gitignore filtering),
+`test/pglite-lock.test.ts` (PGLite lock/concurrency behavior), `test/search-limit.test.ts` (search result count limits),
+`test/cli-action-run.test.ts` (process-level exit-code tests for `gbrain action run` via Bun subprocess + `--preload` fixture),
+`test/fixtures/cli-action-run.preload.ts` (mock module preload for isolated CLI exit-code testing).
 
 E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_URL`.
 - `bun run test:e2e` runs Tier 1 (mechanical, all operations, no API keys)
+- `test/e2e/mechanical.test.ts` runs all operations against real Postgres+pgvector (Tier 1)
+- `test/e2e/mcp.test.ts` verifies MCP server startup and tool definitions (Tier 1, PGLite in-memory)
 - `test/e2e/search-quality.test.ts` runs search quality E2E against PGLite (no API keys, in-memory)
+- `test/e2e/sync.test.ts` tests file sync pipeline against real DB (Tier 1)
 - `test/e2e/upgrade.test.ts` runs check-update E2E against real GitHub API (network required)
 - Tier 2 (`skills.test.ts`) requires OpenClaw + API keys, runs nightly in CI
 - If `.env.testing` doesn't exist in this directory, check sibling worktrees for one:

CONTRIBUTING.md

@@ -41,12 +41,15 @@ src/
     types.ts              Shared types (ActionItem, CommitmentBatch, ExtractionResult)
     action-schema.ts      PGLite DDL + schema init for action_items/action_history tables
     action-engine.ts      Storage layer: CRUD, priority scoring, PGLite lifecycle
-    extractor.ts          LLM commitment extraction with prompt injection defense
+    extractor.ts          LLM commitment extraction (Sonnet default), stabilization, injection defense
+    collector.ts          Wacli message collector: reads WhatsApp exports, checkpoint-aware dedup
+    ingest-runner.ts      Auto-ingest orchestrator: collect → extract → store pipeline, cron-ready
     brief.ts              Morning priority brief generator (ranked + deduped)
-    operations.ts         5 registered ops: action_list/brief/resolve/mark-fp/ingest
+    operations.ts         6 registered ops: action_list/brief/resolve/mark-fp/ingest/ingest-auto
   schema.sql              Postgres DDL
 skills/                   Fat markdown skills for AI agents
 test/                     Unit tests (bun test, no DB required)
+test/fixtures/            Shared test fixtures (mock module preloads for subprocess-level tests)
 test/e2e/                 E2E tests (requires DATABASE_URL, real Postgres+pgvector)
   fixtures/               Miniature realistic brain corpus (16 files)
   helpers.ts              DB lifecycle, fixture import, timing

README.md

@@ -612,6 +612,14 @@ TIMELINE
   gbrain timeline [<slug>]                  View timeline entries
   gbrain timeline-add <slug> <date> <text>  Add timeline entry
 
+ACTION BRAIN (commitment/obligation tracking)
+  gbrain action list [--status S --owner O] List action items (open by default)
+  gbrain action brief                       Morning priority brief from WhatsApp commitments
+  gbrain action resolve <id>                Mark an action item resolved
+  gbrain action mark-fp <id>                Mark extraction as false positive
+  gbrain action ingest [--messages-json J]  Extract and ingest commitments from a message batch
+  gbrain action run                         Auto-ingest pipeline: collect → extract → store (cron-ready)
+
 ADMIN
   gbrain doctor [--json]                    Health checks (pgvector, RLS, schema, embeddings)
   gbrain stats                              Brain statistics

TODOS.md

@@ -102,6 +102,24 @@
 
 **Context:** Identified by adversarial review during v0.10.0 ship.
 
+### Action Brain: extract shared utility module for duplicated pipeline helpers
+**What:** Move `buildCommitmentSourceId`, `toActionTitle`, `resolveSourceMessage`, `resolveSourceMessageId`, `asOptionalNonEmptyString`, `normalizeCommitmentField` into `src/action-brain/utils.ts`. These are currently duplicated across `operations.ts` and `ingest-runner.ts`. Also consolidate `clampConfidence` — operations.ts defaults to 0.5 while ingest-runner.ts defaults to 0, causing inconsistent DB values on edge-case LLM output.
+
+**Why:** DRY violation — 6 utility functions duplicated across files. The clampConfidence inconsistency can silently write different confidence values for the same LLM output depending on the call path.
+
+**Complexity:** Medium — safe refactor, but touches core pipeline code. Test coverage is good.
+
+**Context:** Identified during v0.10.2 ship pre-landing review.
+
+### Action Brain: fix N+1 message lookup in ingest pipeline
+**What:** In `runActionIngest`, `resolveSourceMessage` is called inside a loop over all commitments using `Array.find()`, creating O(commitments × messages) complexity. Pre-build a `Map<messageId, WhatsAppMessage>` before the loop.
+
+**Why:** For typical WhatsApp exports this is fine at MVP scale, but grows quadratically. Easy fix.
+
+**Complexity:** Low — add a `Map` before the store loop in `ingest-runner.ts`.
+
+**Context:** Identified during v0.10.2 ship pre-landing review.
+
 ## Completed
 
 ### Implement AWS Signature V4 for S3 storage backend

VERSION

@@ -1 +1 @@
-0.10.0
+0.10.2

src/action-brain/action-engine.ts

@@ -54,6 +54,11 @@ export interface ActionMutationOptions {
   metadata?: Record<string, unknown>;
 }
 
+export interface CreateActionItemResult {
+  item: ActionItem;
+  created: boolean;
+}
+
 export interface ListActionItemsFilters {
   status?: ActionStatus;
   owner?: string;
@@ -80,6 +85,14 @@ export class ActionEngine {
   constructor(private readonly db: ActionDb) {}
 
   async createItem(input: CreateActionItemInput, options: ActionMutationOptions = {}): Promise<ActionItem> {
+    const result = await this.createItemWithResult(input, options);
+    return result.item;
+  }
+
+  async createItemWithResult(
+    input: CreateActionItemInput,
+    options: ActionMutationOptions = {}
+  ): Promise<CreateActionItemResult> {
     return this.withTransaction(async () => {
       const result = await this.db.query<ActionInsertRow>(
         `WITH inserted AS (
@@ -142,7 +155,10 @@ export class ActionEngine {
         );
       }
 
-      return mapActionItem(row);
+      return {
+        item: mapActionItem(row),
+        created: toBoolean(row.was_inserted),
+      };
     });
   }
 

src/action-brain/brief.ts

@@ -30,10 +30,6 @@ interface ActionItemRow {
   resolved_at: Date | string | null;
 }
 
-interface LastSyncRow {
-  last_sync_at: Date | string | null;
-}
-
 export type BriefContextEnricher = (
   items: ActionItem[]
 ) => Promise<Map<number, string> | Record<number, string>> | Map<number, string> | Record<number, string>;
@@ -99,14 +95,7 @@ export class MorningBriefGenerator {
     if (provided) {
       return ensureDate(provided, 'lastSyncAt');
     }
-
-    const result = await this.db.query<LastSyncRow>(
-      `SELECT max(created_at) AS last_sync_at
-       FROM action_items`
-    );
-
-    const raw = result.rows[0]?.last_sync_at ?? null;
-    return raw ? ensureDate(raw, 'last_sync_at') : null;
+    return null;
   }
 }