Skip to content

feat(localtaskstore): add durable local state for task receipt, outbox, and restart recovery#181

Merged
iAziz786 merged 1 commit intomainfrom
add-hostlink-local-store
Apr 27, 2026
Merged

feat(localtaskstore): add durable local state for task receipt, outbox, and restart recovery#181
iAziz786 merged 1 commit intomainfrom
add-hostlink-local-store

Conversation

@iAziz786
Copy link
Copy Markdown
Contributor

Summary

Add a durable local task store under HOSTLINK_STATE_PATH that persists received, running, final, and interrupted task execution state. This establishes the persistence boundary required for reliable WebSocket reconnect, resend, and restart behavior before task result delivery can be trusted in production.

Changes

New package: app/services/localtaskstore/

  • Configurable SQLite-backed store with spool cap (64MiB default) and terminal reserve (1MiB default)
  • Two-table schema: local_task_executions (task/execution state keyed by task_id + execution_attempt_id) and local_task_outbox_messages (unacked output chunk / final result resend queue)
  • RecordReceived, RecordStarted, AppendOutputChunk, RecordFinal, AckMessage, UnackedMessages, Snapshot, MarkInterruptedRunningTasks operations
  • Oldest-first chunk rotation under spool cap with truncation markers
  • Terminal result preserved under cap pressure
  • Admission control: RecordReceived fails with ErrTerminalReserveUnavailable when outbox cannot reserve space for a future final result
  • Startup recovery: stale running tasks converted to interrupted terminal records with synthetic outbox message
  • Narrow extension interfaces: ReceiptStore, ResultOutbox, RecoveryStore
  • Comprehensive temp-dir crash/restart persistence tests

Modified: config/appconf/

  • LocalTaskStorePath() — derives default from HOSTLINK_STATE_PATH or explicit env override
  • LocalTaskStoreSpoolCapBytes() — env HOSTLINK_LOCAL_STORE_SPOOL_CAP_BYTES, default 64MiB
  • LocalTaskStoreTerminalReserveBytes() — env HOSTLINK_LOCAL_STORE_TERMINAL_RESERVE_BYTES, default 1MiB

Modified: main.go

  • recoverLocalTaskStore() helper: opens store via NewDefault() and runs MarkInterruptedRunningTasks() on boot
  • Store initialized in goroutine after server start, does not block registration, polling, heartbeat, metrics, or self-update

Verification

All tests pass:

  • go test ./config/appconf/
  • go test ./app/services/localtaskstore/
  • go test . (root startup wiring tests)
  • go test ./app/jobs/taskjob ./app/services/taskreporter/

…x, and restart recovery

Add a dedicated SQLite-backed local store under HOSTLINK_STATE_PATH that
persists received, running, final, and interrupted task execution state
keyed by task_id and execution_attempt_id.

Implementation:
- Config helpers for store path, spool cap (default 64MiB), and terminal
  reserve (default 1MiB) with env overrides
- localtaskstore package with RecordReceived, RecordStarted,
  AppendOutputChunk, RecordFinal, AckMessage, UnackedMessages, Snapshot,
  and MarkInterruptedRunningTasks
- Two-table schema: local_task_executions (task/execution state) and
  local_task_outbox_messages (unacked output/final resend queue)
- Spool cap with oldest-first chunk rotation, truncation markers, and
  terminal-preservation semantics
- Startup recovery converts stale running tasks into interrupted terminal
  records
- Narrow ReceiptStore, ResultOutbox, RecoveryStore interfaces for later
  WebSocket adoption
- Hostlink startup wiring in recoverLocalTaskStore() helper
@iAziz786 iAziz786 force-pushed the add-hostlink-local-store branch from 6c23d87 to b17a118 Compare April 27, 2026 12:19
@iAziz786 iAziz786 merged commit fd909cf into main Apr 27, 2026
1 check passed
@iAziz786 iAziz786 deleted the add-hostlink-local-store branch April 27, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant