From 6f05bdaff5b2fc10827f6184627073de9a6fe600 Mon Sep 17 00:00:00 2001 From: norrietaylor Date: Thu, 23 Apr 2026 14:57:11 -0700 Subject: [PATCH 1/3] feat: add local macOS launchd installer with scheduled pipelines MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Installs a supervised Distillery HTTP server as a LaunchAgent plus four scheduled webhook agents (poll every 30 min, classify every 2 h, rescore daily, maintenance weekly) and a weekly image-update agent. All ingestion goes through the running server's /api/* surface — DuckDB is single-writer and the server owns the lock, so sibling CLI processes can't write. Secrets (JINA_API_KEY, DISTILLERY_WEBHOOK_SECRET) live in the macOS login Keychain; the webhook secret is generated on first install. The installer is idempotent and auto-detects OrbStack vs Docker Desktop. Uninstaller preserves the DB and Keychain entries by default. Adds a 'Local macOS (launchd)' page under Getting Started with setup, verification, customization, and troubleshooting. Co-Authored-By: Claude Opus 4.7 --- docs/getting-started/local-macos.md | 166 ++++++++++++ mkdocs.yml | 1 + scripts/local-macos/README.md | 48 ++++ scripts/local-macos/install.sh | 239 ++++++++++++++++++ .../local-macos/templates/_webhook_common.sh | 46 ++++ scripts/local-macos/templates/classify.sh | 8 + scripts/local-macos/templates/distillery.yaml | 40 +++ .../launchd/local.distillery-classify.plist | 28 ++ .../local.distillery-maintenance.plist | 39 +++ .../launchd/local.distillery-poll.plist | 28 ++ .../launchd/local.distillery-rescore.plist | 36 +++ .../launchd/local.distillery-update.plist | 38 +++ .../templates/launchd/local.distillery.plist | 36 +++ scripts/local-macos/templates/maintenance.sh | 9 + scripts/local-macos/templates/poll.sh | 10 + scripts/local-macos/templates/rescore.sh | 7 + scripts/local-macos/templates/run.sh | 51 ++++ scripts/local-macos/templates/update.sh | 45 ++++ scripts/local-macos/uninstall.sh | 122 +++++++++ 19 files changed, 997 insertions(+) create mode 100644 docs/getting-started/local-macos.md create mode 100644 scripts/local-macos/README.md create mode 100755 scripts/local-macos/install.sh create mode 100644 scripts/local-macos/templates/_webhook_common.sh create mode 100644 scripts/local-macos/templates/classify.sh create mode 100644 scripts/local-macos/templates/distillery.yaml create mode 100644 scripts/local-macos/templates/launchd/local.distillery-classify.plist create mode 100644 scripts/local-macos/templates/launchd/local.distillery-maintenance.plist create mode 100644 scripts/local-macos/templates/launchd/local.distillery-poll.plist create mode 100644 scripts/local-macos/templates/launchd/local.distillery-rescore.plist create mode 100644 scripts/local-macos/templates/launchd/local.distillery-update.plist create mode 100644 scripts/local-macos/templates/launchd/local.distillery.plist create mode 100644 scripts/local-macos/templates/maintenance.sh create mode 100644 scripts/local-macos/templates/poll.sh create mode 100644 scripts/local-macos/templates/rescore.sh create mode 100644 scripts/local-macos/templates/run.sh create mode 100644 scripts/local-macos/templates/update.sh create mode 100755 scripts/local-macos/uninstall.sh diff --git a/docs/getting-started/local-macos.md b/docs/getting-started/local-macos.md new file mode 100644 index 00000000..80220617 --- /dev/null +++ b/docs/getting-started/local-macos.md @@ -0,0 +1,166 @@ +# Local macOS (launchd) + +Runs a supervised Distillery server on your Mac with scheduled ingestion, image updates, and maintenance — zero cloud, zero CI. Installed by `scripts/local-macos/install.sh`. + +This is a different profile from the [stdio local setup](local-setup.md): + +| | stdio (`local-setup.md`) | launchd (this page) | +|---|---|---| +| Transport | stdio | HTTP on `127.0.0.1:8000` | +| Lifecycle | launched per Claude Code session | supervised daemon, runs 24/7 | +| Scheduled work | optional Claude Code routines | six LaunchAgents | +| Image updates | manual | weekly auto-pull | +| Good for | single-user, low-footprint | heavier feed ingestion, personal KB | + +## What gets installed + +Six LaunchAgents in `~/Library/LaunchAgents/`: + +| Label | Cadence | Purpose | +|---|---|---| +| `local.distillery` | supervised | Runs the GHCR container (`ghcr.io/norrietaylor/distillery:latest`) in the foreground; launchd restarts it on exit. | +| `local.distillery-update` | Mondays 09:00 | `docker pull` the `:latest` image; if the digest changed, kickstart the server. | +| `local.distillery-poll` | every 30 min | `POST /api/poll` — fetch new items from configured feed sources. | +| `local.distillery-classify` | every 2 hours | `POST /api/hooks/classify-batch` — batch-classify pending inbox entries. | +| `local.distillery-rescore` | daily 04:15 | `POST /api/rescore` — refresh feed-entry relevance scores. | +| `local.distillery-maintenance` | Mondays 05:00 | `POST /api/maintenance` — orchestrated `poll → rescore → classify-batch`. | + +Plus these files in `~/.distillery/`: + +- `distillery.yaml` — server config +- `run.sh`, `update.sh` — server supervisor + image-update worker +- `poll.sh`, `classify.sh`, `rescore.sh`, `maintenance.sh` — webhook workers +- `_webhook_common.sh` — shared helper sourced by the four workers +- `distillery.db` — the DuckDB file +- `*.log` — per-agent stdout/stderr + +And two entries in the macOS login Keychain (`security` CLI): + +- `JINA_API_KEY` — embedding provider key +- `DISTILLERY_WEBHOOK_SECRET` — bearer token for `/api/*` routes + +## Prerequisites + +- macOS (tested on Sonoma and later) +- [OrbStack](https://orbstack.dev) *or* Docker Desktop, running +- A free [Jina AI](https://jina.ai) API key + +Docker on Apple Silicon runs the container under Rosetta (`linux/amd64`). No separate arm64 image is published. + +## Install + +```bash +git clone https://github.com/norrietaylor/distillery.git +cd distillery +./scripts/local-macos/install.sh +``` + +The installer will: + +1. Verify Docker is reachable. +2. Prompt for your Jina API key (skipped if already in the Keychain). +3. Generate a 32-byte webhook bearer secret (skipped if already in the Keychain). +4. Write the config, scripts, and plists. +5. Bootstrap every agent and kickstart the server. +6. Wait up to 30 s for `http://127.0.0.1:8000/` to respond. + +Re-run any time to refresh scripts/plists — the installer is idempotent and never touches the database or existing secrets. + +### Flags + +| Flag | Effect | +|---|---| +| `--jina-key ` | Pass the Jina key non-interactively (useful for scripted installs). | +| `--no-kickstart` | Load the agents but don't start the server or poll for readiness. | + +## Point Claude Code at the server + +Add to `~/.claude/settings.json`: + +```json +{ + "mcpServers": { + "distillery": { + "type": "http", + "url": "http://127.0.0.1:8000/mcp" + } + } +} +``` + +Restart Claude Code, then run `/setup` — it will detect the running server and offer to configure the reporting routines (feed health check, stale check, weekly digest). Those routines run inside Claude Code and complement the ingestion agents installed here. + +## Verify + +```bash +# All six agents loaded +launchctl list | grep distillery + +# Kick the server in case it's not running +launchctl kickstart -k gui/$(id -u)/local.distillery + +# Server is up +curl -sS -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8000/ + +# Webhook auth works (expects 202 Accepted) +SECRET=$(security find-generic-password -a "$USER" -s DISTILLERY_WEBHOOK_SECRET -w) +curl -sS -H "Authorization: Bearer $SECRET" -X POST http://127.0.0.1:8000/api/poll +``` + +Tail the logs to watch the agents fire: + +```bash +tail -F ~/.distillery/{poll,classify,rescore,maintenance}.out.log +``` + +## Customizing + +### Change the cadence + +Edit the `StartInterval` (seconds) or `StartCalendarInterval` block in the plist, then reload: + +```bash +launchctl bootout gui/$(id -u)/local.distillery-poll +launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/local.distillery-poll.plist +``` + +### Rotate the webhook secret + +```bash +security delete-generic-password -a "$USER" -s DISTILLERY_WEBHOOK_SECRET +./scripts/local-macos/install.sh # regenerates on next install +launchctl kickstart -k gui/$(id -u)/local.distillery +``` + +### Add a feed source + +Open Claude Code and run `/watch add `, or call `distillery_watch` directly via the MCP. + +### Disable one pipeline + +```bash +launchctl bootout gui/$(id -u)/local.distillery-rescore +rm ~/Library/LaunchAgents/local.distillery-rescore.plist +``` + +## Troubleshooting + +**Webhook returns 401.** The server didn't see `DISTILLERY_WEBHOOK_SECRET` at startup — check the Keychain entry exists, then `launchctl kickstart -k gui/$(id -u)/local.distillery`. + +**Webhook returns 429 "too_early".** Per-endpoint cooldown. The worker treats this as success; nothing to fix. + +**Container won't start.** Inspect `~/.distillery/server.err.log`. The supervisor exits 1 if `JINA_API_KEY` or `DISTILLERY_WEBHOOK_SECRET` is missing from the Keychain — launchd respects `ThrottleInterval=10` so it retries every 10 s. + +**`IO Error: Conflicting lock`.** Something other than the server is trying to open the DuckDB file (`docker exec ... distillery `, a second container, a CLI on the host). DuckDB is single-writer; only the running server process may open the DB for writes. Use the HTTP webhooks instead. + +**Agents silently not firing.** `launchctl print gui/$(id -u)/