diff --git a/.ccb/.claude-session b/.ccb/.claude-session new file mode 100644 index 00000000..037f0bc9 --- /dev/null +++ b/.ccb/.claude-session @@ -0,0 +1,16 @@ +{ + "session_id": "ai-1775888498-61737", + "ccb_project_id": "ea66bbb87cd797287d668bb57a8ab9762f0509c76e37e90cb1e0536d75ec7b72", + "work_dir": "/Users/hansonmei/Projects/nowledge-community", + "work_dir_norm": "/Users/hansonmei/Projects/nowledge-community", + "start_dir": "/Users/hansonmei/Projects/nowledge-community", + "terminal": "tmux", + "active": false, + "started_at": "2026-04-11 14:21:38", + "updated_at": "2026-04-13 14:04:31", + "pane_id": "%7", + "pane_title_marker": "CCB-Claude-ea66bbb8", + "claude_session_path": "/Users/hansonmei/.claude/projects/-Users-hansonmei-Projects-nowledge-community/3d8de919-0380-474d-9fef-24b3612d1289.jsonl", + "claude_session_id": "3d8de919-0380-474d-9fef-24b3612d1289", + "ended_at": "2026-04-13 14:04:31" +} \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..01e8037b --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,30 @@ +# Nowledge Community — Agent Guidelines + +## Registry + +[`integrations.json`](integrations.json) is the **single source of truth** for all Nowledge Mem integrations. It tracks capabilities, versions, install commands, transport, tool naming, and thread save methods. + +**When adding or modifying any integration, update `integrations.json` first.** Other surfaces (website `integrations.ts`, desktop app integrations view, README tables, marketplace JSONs) derive from or validate against this file. + +The desktop app fetches this file at runtime from `https://raw.githubusercontent.com/nowledge-co/community/main/integrations.json` for plugin update awareness. Changes to the schema (adding/removing/renaming fields) affect: +- **Rust** (`lib.rs`): `fetch_plugin_registry`, `detect_installed_plugins`, `write_plugin_update_state` +- **TypeScript** (`plugin-update-manager.ts`): `RegistryIntegration` interface +- **Python** (`health.py`): `_read_plugin_update_state` reader + +## Behavioral Guidance + +[`shared/behavioral-guidance.md`](shared/behavioral-guidance.md) defines when plugins should search, save, read Working Memory, and distill. All plugins should align with this shared guidance. + +## Plugin Development + +See [`docs/PLUGIN_DEVELOPMENT_GUIDE.md`](docs/PLUGIN_DEVELOPMENT_GUIDE.md) for authoring rules, directory layout, and testing expectations. + +## Submodules + +`nowledge-mem-gemini-cli` is a nested submodule (separate repo with its own release cycle). All other integrations are normal directories in this repo. + +## Commit Workflow + +When modifying this repo as a submodule of the parent `muscat` repo: +1. Commit inside `community/` first +2. Then stage the updated submodule reference in the parent repo diff --git a/README.md b/README.md index a011d7e1..dd930958 100644 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ Each directory is a standalone integration. Pick the one that matches your tool. | **[Antigravity Trajectory Extractor](https://github.com/jijiamoer/antigravity-trajectory-extractor)** | `git clone https://github.com/jijiamoer/antigravity-trajectory-extractor.git` | Live RPC extraction for Antigravity conversation trajectories. | | **[Windsurf Trajectory Extractor](https://github.com/jijiamoer/windsurf-trajectory-extractor)** | `git clone https://github.com/jijiamoer/windsurf-trajectory-extractor.git` | Offline protobuf extraction for Windsurf Cascade conversation history. | | **[Cursor Plugin](nowledge-mem-cursor-plugin)** | Link `nowledge-mem-cursor-plugin` into `~/.cursor/plugins/local/nowledge-mem-cursor` | Cursor-native plugin package with a session-start Working Memory hook, bundled MCP config, rules, and honest `save-handoff` semantics. | -| **[Codex Plugin](nowledge-mem-codex-plugin)** | Copy the full plugin directory, including `.codex-plugin`, to `~/.codex/plugins/cache/local/nowledge-mem/local/` and enable it in `~/.codex/config.toml` | Packaged Codex skills for Working Memory bootstrap, proactive recall guidance, real session save, and distillation. | +| **[Codex Plugin](nowledge-mem-codex-plugin)** | Copy the full plugin directory, including `.codex-plugin`, to `~/.codex/plugins/cache/local/nowledge-mem/local/`, enable it in `~/.codex/config.toml`, then run the bundled hook installer | Packaged Codex skills for Working Memory bootstrap, proactive recall guidance, and distillation, plus an optional host-level `Stop` hook installer for automatic transcript capture. | | **[OpenClaw Plugin](nowledge-mem-openclaw-plugin)** | `openclaw plugins install clawhub:@nowledge/openclaw-nowledge-mem` | Full memory lifecycle with memory tools, thread tools, automatic capture, and distillation. | | **[Alma Plugin](nowledge-mem-alma-plugin)** | Search Nowledge in Alma official Plugin marketplace | Alma-native plugin with Working Memory, thread-aware recall, structured saves, and optional auto-capture. | | **[Bub Plugin](nowledge-mem-bub-plugin)** | `pip install nowledge-mem-bub` | Bub-native plugin: cross-tool knowledge, auto-capture via save_state, Working Memory, and graph exploration. | diff --git a/docs/PLUGIN_DEVELOPMENT_GUIDE.md b/docs/PLUGIN_DEVELOPMENT_GUIDE.md index ba41c17e..66fcb738 100644 --- a/docs/PLUGIN_DEVELOPMENT_GUIDE.md +++ b/docs/PLUGIN_DEVELOPMENT_GUIDE.md @@ -119,7 +119,7 @@ All behavioral heuristics (when to search, when to save, when to read Working Me ### Skill naming -Skill names use kebab-case and are consistent across all plugins: +Skill names use kebab-case. Prefer the shared names below for new integrations, but preserve published compatibility exceptions: | Skill | Purpose | |-------|---------| @@ -131,6 +131,9 @@ Skill names use kebab-case and are consistent across all plugins: | `check-integration` | Detect agent, verify setup, guide plugin installation | | `status` | Connection and configuration diagnostics | +Published exception: +- Codex already shipped `working-memory`; keep that name for Codex unless and until a compatible alias is introduced. + ### Autonomous save is required Every integration's distill/save guidance MUST include proactive save encouragement: diff --git a/integrations.json b/integrations.json index 1fc507f8..c4fb38fe 100644 --- a/integrations.json +++ b/integrations.json @@ -110,7 +110,7 @@ "search": true, "distill": true, "autoRecall": false, - "autoCapture": false, + "autoCapture": true, "graphExploration": false, "status": true }, @@ -123,23 +123,24 @@ "bootstrap": "guided", "recall": "guided", "distill": "guided", - "threads": "explicit-save", + "threads": "automatic-capture", "bestResultRequires": [ - "Install the Codex package and enable plugins in ~/.codex/config.toml", + "Install the Codex package, enable plugins in ~/.codex/config.toml, then run the bundled hook installer", "Keep nmem available on this machine", + "Run the hook installer with a Python interpreter that can import nmem_cli", "Merge the package AGENTS.md into the project for stronger follow-through" ] }, "install": { - "command": "mkdir -p ~/.codex/plugins/cache/local/nowledge-mem/local && cp -R nowledge-mem-codex-plugin/. ~/.codex/plugins/cache/local/nowledge-mem/local/", - "updateCommand": "cp -R nowledge-mem-codex-plugin/. ~/.codex/plugins/cache/local/nowledge-mem/local/", + "command": "mkdir -p ~/.codex/plugins/cache/local/nowledge-mem/local && cp -R nowledge-mem-codex-plugin/. ~/.codex/plugins/cache/local/nowledge-mem/local/ && python3 ~/.codex/plugins/cache/local/nowledge-mem/local/scripts/install_hooks.py", + "updateCommand": "cp -R nowledge-mem-codex-plugin/. ~/.codex/plugins/cache/local/nowledge-mem/local/ && python3 ~/.codex/plugins/cache/local/nowledge-mem/local/scripts/install_hooks.py", "configRequired": "[features] plugins = true AND [plugins.\"nowledge-mem@local\"] enabled = true in ~/.codex/config.toml", "detectionHint": "Running as Codex CLI agent; ~/.codex/ exists", "docsUrl": "/docs/integrations/codex-cli" }, "toolNaming": { "convention": "cli-direct", - "note": "Packaged Codex skills teach the agent to invoke nmem CLI progressively. Working Memory is the reliable bootstrap; search and distill remain model-driven unless repo guidance tightens policy." + "note": "Packaged Codex skills teach the agent to invoke nmem CLI progressively. Working Memory is the reliable bootstrap; search and distill remain model-driven unless repo guidance tightens policy. The bundled host-level Stop hook installer adds automatic transcript capture once installed." }, "skills": ["working-memory", "search-memory", "save-thread", "distill-memory", "status"], "slashCommands": [] diff --git a/nowledge-mem-codex-plugin/CHANGELOG.md b/nowledge-mem-codex-plugin/CHANGELOG.md index dd871297..eeb42324 100644 --- a/nowledge-mem-codex-plugin/CHANGELOG.md +++ b/nowledge-mem-codex-plugin/CHANGELOG.md @@ -1,6 +1,11 @@ # Changelog ## [0.1.3] - 2026-04-11 +### Added + +- **Bundled Codex hook installer**: added `scripts/install_hooks.py` to install a native Codex `Stop` hook into `~/.codex/hooks.json`, copy the runtime hook into `~/.codex/hooks/`, and enable `codex_hooks = true` in `~/.codex/config.toml`. +- **Host-level Stop hook runtime**: added `hooks/nmem-stop-save.py`, which reads Codex's `transcript_path`, parses the rollout directly with `nmem_cli.session_import.parse_codex_session_streaming`, and imports the current thread into Mem without relying on `nmem t save --from codex` session discovery. +- **Plugin validation and release notes**: added `scripts/validate-plugin.mjs` and `RELEASING.md` so the Codex package now has the same explicit release contract as the other maintained integrations in this repo. ### Improved @@ -11,7 +16,9 @@ ### Fixed - **Install copy command**: Codex install/update instructions now preserve hidden files such as `.codex-plugin/plugin.json`. -- **Docs honesty**: removed wording that implied Codex has lifecycle-hook automation comparable to hosts like Claude Code or OpenClaw. +- **Hook runtime now uses the installer interpreter**: `scripts/install_hooks.py` now pins the copied `Stop` hook to the same Python interpreter that ran the installer, and it fails fast if that interpreter cannot import `nmem_cli`. +- **Refresh helper no longer crashes on import**: `scripts/refresh_thread_titles.py` now lazily loads `nmem_cli` and counts generic refresh failures instead of aborting the full run. +- **Docs honesty**: clarify that Codex memory search and distill remain skill-guided, while automatic transcript capture comes from the optional host-level hook installer. ## [0.1.2] - 2026-04-06 diff --git a/nowledge-mem-codex-plugin/README.md b/nowledge-mem-codex-plugin/README.md index 078f1e14..d39279dc 100644 --- a/nowledge-mem-codex-plugin/README.md +++ b/nowledge-mem-codex-plugin/README.md @@ -9,6 +9,7 @@ Switch between Claude Code, Gemini, Cursor, and Codex without losing context. De - **Pick up where you left off.** Every session can start from your current priorities, recent decisions, and unresolved questions. - **Search when prior work matters.** The plugin teaches Codex when to search memories and threads, especially on continuation-style tasks. - **Insights stick around.** The plugin teaches Codex to distill durable decisions and learnings when they emerge. +- **Automatic transcript capture is available.** Once you run the bundled host-level `Stop` hook installer, Codex can import the real transcript after each completed turn. - **Real session history.** Save the full Codex transcript, not just a summary. - **Quick diagnostics.** One command to verify everything is connected. @@ -26,7 +27,9 @@ Codex does not give this package hard lifecycle hooks like Claude Code or OpenCl ## Prerequisites -`nmem` CLI must be in your PATH. +For day-to-day skill usage, put `nmem` on your PATH. + +For automatic `Stop`-hook capture, the Python interpreter that runs `scripts/install_hooks.py` must be able to import `nmem_cli`. `uvx --from nmem-cli nmem` is enough for interactive CLI commands, but it does not make `nmem_cli` importable to Codex's hook runtime by itself. **Quickest path** (if the Nowledge Mem desktop app is running): Settings > Preferences > Developer Tools > Install CLI @@ -36,9 +39,11 @@ Settings > Preferences > Developer Tools > Install CLI ```bash curl -LsSf https://astral.sh/uv/install.sh | sh uvx --from nmem-cli nmem --version +python3 -m pip install nmem-cli +python3 -c "import nmem_cli, nmem_cli.session_import" ``` -Or `pip install nmem-cli`. Then verify with `nmem status`. +Then verify with `nmem status`. ## Install @@ -62,6 +67,12 @@ plugins = true enabled = true ``` +Install the bundled Codex hook helper with the same Python you want Codex to use for the hook runtime: + +```bash +python3 ~/.codex/plugins/cache/local/nowledge-mem/local/scripts/install_hooks.py +``` + Restart Codex after installation. ### Repo-level (this project only) @@ -100,6 +111,31 @@ The path is relative to the repo root, not to the `marketplace.json` file. You still need the feature gate and plugin entry in `~/.codex/config.toml` (see Home-level above). Codex will discover the marketplace on startup and load the plugin from the repo-local source. +If you want automatic thread capture in repo-level installs too, run: + +```bash +python3 ./.agents/nowledge-mem/scripts/install_hooks.py +``` + +## Codex Hook Support + +Codex hooks are currently **host-level**, not plugin-local. In practice that means Codex reads `~/.codex/hooks.json`, not hook assets tucked inside the plugin directory. + +This package therefore ships a small helper installer: + +```bash +python3 ~/.codex/plugins/cache/local/nowledge-mem/local/scripts/install_hooks.py +``` + +It does four things: + +1. Copies the bundled runtime hook into `~/.codex/hooks/nowledge-mem-stop-save.py` +2. Merges a `Stop` hook entry into `~/.codex/hooks.json` +3. Ensures `codex_hooks = true` exists under `[features]` in `~/.codex/config.toml` +4. Pins the installed hook shebang to the current `sys.executable` so Codex runs the hook with the same Python interpreter you used for installation + +The installed `Stop` hook reads Codex's `transcript_path`, parses the rollout directly with `nmem_cli.session_import.parse_codex_session_streaming`, and imports the current thread into Mem. It keeps lightweight state in `~/.codex/nowledge_mem_codex_hook_state.json` so repeated `Stop` events only re-import when the transcript actually changed. + ## Verify Start a new Codex session and ask: "What was I working on?" The agent should load your Working Memory briefing. @@ -108,6 +144,15 @@ Then test one continuation-style prompt such as "What did we decide before about If Mem is not running yet, try `$nowledge-mem:status` to check connectivity. +For automatic capture, you can also run a tiny one-shot check: + +```bash +codex exec -C . "Reply with exactly OK and nothing else." +tail -n 20 ~/.codex/log/nowledge-mem-stop-hook.log +``` + +You should see `start event=Stop`, `nmem_exit=0`, and a `created thread=...`, `appended thread=...`, or `skip: ...` outcome entry in the hook log. + ## Update ```bash @@ -115,6 +160,7 @@ git clone https://github.com/nowledge-co/community.git /tmp/nowledge-community-u cp -R /tmp/nowledge-community-update/nowledge-mem-codex-plugin/. \ ~/.codex/plugins/cache/local/nowledge-mem/local/ rm -rf /tmp/nowledge-community-update +python3 ~/.codex/plugins/cache/local/nowledge-mem/local/scripts/install_hooks.py ``` Restart Codex after updating. @@ -165,9 +211,12 @@ If you used `nowledge-mem-codex-prompts` before: ## Troubleshooting - **"Command not found: nmem"**: `pip install nmem-cli` or use `uvx --from nmem-cli nmem`. See [Getting Started](https://mem.nowledge.co/docs/installation). -- **"Cannot connect to server"**: Run `nmem status`. For remote setups, check `~/.nowledge-mem/config.json`. See [Remote Access](https://mem.nowledge.co/docs/remote-access). +- **"Cannot connect to server"**: Run `nmem status`. For remote setups, check `nmem config client show`. See [Remote Access](https://mem.nowledge.co/docs/remote-access). - **Skills not appearing**: Restart Codex after installing. Verify both `[features] plugins = true` and `[plugins."nowledge-mem@local"] enabled = true` are in `~/.codex/config.toml`. - **"plugin is not installed"**: Check that the plugin files are at `~/.codex/plugins/cache/local/nowledge-mem/local/` and that `.codex-plugin/plugin.json` exists inside that directory. +- **Hooks do not fire**: Run `python3 ~/.codex/plugins/cache/local/nowledge-mem/local/scripts/install_hooks.py` again, then confirm `~/.codex/hooks.json` exists and `codex_hooks = true` is present under `[features]` in `~/.codex/config.toml`. +- **Installer says `nmem_cli` is missing**: Run `python3 -m pip install nmem-cli`, then verify `python3 -c "import nmem_cli, nmem_cli.session_import"` succeeds before rerunning `scripts/install_hooks.py`. +- **No auto-save after a response**: Inspect `~/.codex/log/nowledge-mem-stop-hook.log`. The hook imports directly from Codex's `transcript_path`; if the log shows repeated skips, check that the installed hook starts with the same `#!python` you used for `scripts/install_hooks.py`, and that `python3 -c "import nmem_cli, nmem_cli.session_import"` succeeds in that interpreter. - **Only Working Memory runs, but search/distill never show up**: this package is skill-guided, not hook-driven. Merge the package `AGENTS.md` into the project root for stronger repo-specific behavior, and verify you are asking a continuation-style question rather than a fresh isolated one. ## Links diff --git a/nowledge-mem-codex-plugin/RELEASING.md b/nowledge-mem-codex-plugin/RELEASING.md new file mode 100644 index 00000000..fd10dcb5 --- /dev/null +++ b/nowledge-mem-codex-plugin/RELEASING.md @@ -0,0 +1,46 @@ +# Releasing Nowledge Mem for Codex + +This package has two surfaces: + +1. the Codex plugin itself under `.codex-plugin/` and `skills/` +2. the optional host-level hook installer under `scripts/install_hooks.py` + +Codex does not currently load hook assets directly from the plugin directory. The installer bridges that gap by copying the bundled runtime hook into `~/.codex/hooks/`, merging `~/.codex/hooks.json`, and enabling `codex_hooks = true` in `~/.codex/config.toml`. + +## Preflight + +Run from the repository root: + +```bash +node nowledge-mem-codex-plugin/scripts/validate-plugin.mjs +``` + +## Release Checklist + +- bump `.codex-plugin/plugin.json` version +- add a top entry to `CHANGELOG.md` +- bump `integrations.json` -> `integrations[id="codex-cli"].version` +- update expected version checks in `scripts/validate-plugin.mjs` +- keep install examples on `cp -r .../. ...` so `.codex-plugin/` is copied +- keep `scripts/install_hooks.py` idempotent +- keep the installer enforcing a Python runtime that can `import nmem_cli` +- keep the copied hook pinned to the installer interpreter (`sys.executable`) +- keep `hooks/nmem-stop-save.py` focused on direct transcript import via `transcript_path` +- re-run `node nowledge-mem-codex-plugin/scripts/validate-plugin.mjs` + +## Manual Smoke Test + +After copying the plugin into `~/.codex/plugins/cache/local/nowledge-mem/local/`: + +```bash +python3 -c "import nmem_cli, nmem_cli.session_import" +python3 ~/.codex/plugins/cache/local/nowledge-mem/local/scripts/install_hooks.py +codex exec -C . "Reply with exactly OK and nothing else." +tail -n 20 ~/.codex/log/nowledge-mem-stop-hook.log +``` + +Expect: + +- `start event=Stop` +- `nmem_exit=0` +- a `created thread=...`, `appended thread=...`, or `skip: ...` outcome line in the hook log diff --git a/nowledge-mem-codex-plugin/hooks/nmem-stop-save.py b/nowledge-mem-codex-plugin/hooks/nmem-stop-save.py new file mode 100644 index 00000000..7ba67425 --- /dev/null +++ b/nowledge-mem-codex-plugin/hooks/nmem-stop-save.py @@ -0,0 +1,324 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import fcntl +import hashlib +import json +import os +import re +import sys +import tempfile +import time +from contextlib import contextmanager +from datetime import datetime, timezone +from pathlib import Path +from urllib.parse import quote + + +HOME = Path.home() +CODEX_DIR = HOME / ".codex" +LOG_FILE = CODEX_DIR / "log" / "nowledge-mem-stop-hook.log" +STATE_FILE = CODEX_DIR / "nowledge_mem_codex_hook_state.json" +STATE_LOCK_FILE = CODEX_DIR / "nowledge_mem_codex_hook_state.lock" +LOCK_TIMEOUT_SECONDS = 2.0 + + +def ensure_parent(path: Path) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + + +def log(message: str) -> None: + ensure_parent(LOG_FILE) + timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S") + with LOG_FILE.open("a", encoding="utf-8") as handle: + handle.write(f"[{timestamp}] {message}\n") + + +def load_json(path: Path) -> dict: + if not path.exists(): + return {} + try: + payload = json.loads(path.read_text(encoding="utf-8")) + return payload if isinstance(payload, dict) else {} + except Exception: + return {} + + +def save_json(path: Path, payload: dict) -> None: + ensure_parent(path) + temp_path: Path | None = None + try: + with tempfile.NamedTemporaryFile( + "w", + delete=False, + dir=path.parent, + encoding="utf-8", + ) as handle: + json.dump(payload, handle, ensure_ascii=False, indent=2, sort_keys=True) + temp_path = Path(handle.name) + os.replace(temp_path, path) + finally: + if temp_path is not None and temp_path.exists(): + temp_path.unlink(missing_ok=True) + + +@contextmanager +def state_lock( + lock_path: Path = STATE_LOCK_FILE, + timeout_seconds: float = LOCK_TIMEOUT_SECONDS, +): + ensure_parent(lock_path) + with lock_path.open("a+", encoding="utf-8") as handle: + deadline = time.monotonic() + timeout_seconds + while True: + try: + fcntl.flock(handle.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) + break + except BlockingIOError: + if time.monotonic() >= deadline: + raise TimeoutError(f"timed out waiting for lock {lock_path}") + time.sleep(0.05) + + try: + yield + finally: + fcntl.flock(handle.fileno(), fcntl.LOCK_UN) + + +def configure_nmem_env() -> None: + config_path = HOME / ".nowledge-mem" / "config.json" + config = load_json(config_path) + api_url = config.get("apiUrl") or config.get("api_url") + api_key = config.get("apiKey") or config.get("api_key") + if api_url and "NMEM_API_URL" not in os.environ: + os.environ["NMEM_API_URL"] = str(api_url) + if api_key and "NMEM_API_KEY" not in os.environ: + os.environ["NMEM_API_KEY"] = str(api_key) + + +def shorten_title(text: str, limit: int = 60) -> str: + normalized = text.replace("\n", " ").strip() + if len(normalized) <= limit: + return normalized + return normalized[:limit].rstrip() + "..." + + +def is_agents_preamble(text: str) -> bool: + stripped = text.strip() + return ( + stripped.startswith("# AGENTS.md instructions for ") + or ( + stripped.startswith("# AGENTS.md") + and ( + "" in stripped + or "\n## Purpose" in stripped + or "\n## nmem" in stripped + ) + ) + ) + + +def is_environment_context(text: str) -> bool: + stripped = text.strip() + return stripped.startswith("") and stripped.endswith("") + + +def extract_files_request(text: str) -> str | None: + marker = "## My request for Codex" + if marker not in text: + return None + tail = text.split(marker, 1)[1] + tail = re.sub(r"^[:：\s]+", "", tail) + return tail.strip() or None + + +def summarize_user_message(text: str) -> str: + normalized = text.replace("\n", " ").strip() + normalized = re.sub( + r"^codex://threads/[0-9a-fA-F-]+[\s,，:：-]*", + "", + normalized, + ) + normalized = re.sub( + r"^(请帮我|请你|请|麻烦你|麻烦|帮我看看|你看看|帮我看下|帮我看一下|帮我|看下|看一下)\s*", + "", + normalized, + ) + normalized = re.sub( + r"^(can you|could you|please|help me|take a look at|look into)\s+", + "", + normalized, + flags=re.IGNORECASE, + ) + normalized = normalized.replace("是不是", "是否").replace("能不能", "是否能") + normalized = normalized.strip(" \t\r\n?？!！。") + normalized = re.sub(r"\s+", " ", normalized) + return shorten_title(normalized or text) + + +def derive_thread_title(parsed: dict, cwd: str) -> str: + for message in parsed.get("messages", []): + if message.get("role") != "user": + continue + content = (message.get("content") or "").strip() + if not content or is_agents_preamble(content) or is_environment_context(content): + continue + files_request = extract_files_request(content) + if files_request: + return summarize_user_message(files_request) + return summarize_user_message(content) + + workspace = cwd or parsed.get("workspace") or "" + if workspace: + return f"Codex: {Path(workspace).name}" + + parsed_title = (parsed.get("title") or "").strip() + if parsed_title and not is_agents_preamble(parsed_title): + return shorten_title(parsed_title) + + return "Codex Session" + + +def import_current_transcript(payload: dict) -> tuple[int, str]: + session_id = payload.get("session_id") or "" + raw_transcript_path = payload.get("transcript_path") or "" + cwd = payload.get("cwd") or "" + hook_event_name = payload.get("hook_event_name") or "unknown" + + log(f"start event={hook_event_name} session={session_id or 'missing'} cwd={cwd or 'missing'}") + if not session_id: + return 0, "skip: missing session_id" + if not raw_transcript_path: + return 0, f"skip: transcript_path missing or unreadable for session={session_id}" + + transcript_path = Path(raw_transcript_path) + if not (transcript_path.exists() and transcript_path.is_file()): + return 0, f"skip: transcript_path missing or unreadable for session={session_id}" + log(f"transcript={transcript_path}") + + configure_nmem_env() + try: + from nmem_cli.cli import api_get_optional, api_post + from nmem_cli.session_import import parse_codex_session_streaming + except Exception as exc: + return 0, f"skip: failed to import nmem_cli modules: {exc}" + + try: + parsed = parse_codex_session_streaming(transcript_path, truncate_large_content=True) + except Exception as exc: + return 0, f"skip: failed to parse codex rollout: {exc}" + + messages = [ + {"role": message["role"], "content": message.get("content", "")} + for message in parsed.get("messages", []) + if isinstance(message, dict) and message.get("role") + ] + if not messages: + return 0, f"skip: parsed zero messages for session={session_id}" + + thread_id = parsed.get("thread_id") or f"codex-{session_id}" + message_count = len(messages) + content_hash = hashlib.sha256( + json.dumps(messages, ensure_ascii=False, sort_keys=True).encode("utf-8") + ).hexdigest() + + title = derive_thread_title(parsed, cwd) + workspace = parsed.get("workspace") or cwd or None + project = Path(cwd).name if cwd else None + metadata = parsed.get("metadata") or {} + + try: + with state_lock(): + state = load_json(STATE_FILE) + previous = state.get(session_id, {}) + if ( + previous.get("content_hash") == content_hash + and previous.get("message_count") == message_count + ): + return 0, f"skip: unchanged transcript for session={session_id}" + + import_messages = messages + previous_count = previous.get("message_count") + if ( + isinstance(previous_count, int) + and previous_count > 0 + and previous_count < message_count + and previous.get("thread_id") == thread_id + and previous.get("source_file") == str(transcript_path) + ): + import_messages = messages[previous_count:] + + if not import_messages: + return 0, f"skip: no new messages for session={session_id}" + + encoded_thread_id = quote(thread_id, safe='') + existing_thread = api_get_optional(f"/threads/{encoded_thread_id}") + + if existing_thread is None: + api_post( + "/threads", + { + "thread_id": thread_id, + "title": title, + "messages": messages, + "source": "codex", + "project": project, + "workspace": workspace, + "metadata": metadata, + }, + ) + action = "created" + synced_messages = len(messages) + else: + api_post( + f"/threads/{encoded_thread_id}/append", + { + "messages": import_messages, + "deduplicate": True, + }, + ) + action = "appended" + synced_messages = len(import_messages) + + with state_lock(): + state = load_json(STATE_FILE) + state[session_id] = { + "content_hash": content_hash, + "message_count": message_count, + "thread_id": thread_id, + "source_file": str(transcript_path), + "updated_at": datetime.now(timezone.utc).isoformat(), + } + save_json(STATE_FILE, state) + except TimeoutError as exc: + return 0, f"skip: failed to lock state file for session={session_id}: {exc}" + except Exception as exc: + return 0, f"skip: failed to sync thread={thread_id}: {exc}" + + return 0, f"{action}: thread={thread_id} synced_messages={synced_messages} total_messages={message_count}" + + +def main() -> int: + raw = sys.stdin.read() + if not raw.strip(): + log("skip: empty hook payload") + return 0 + + try: + payload = json.loads(raw) + except Exception as exc: + log(f"skip: invalid hook payload: {exc}") + return 0 + + configure_nmem_env() + status, output = import_current_transcript(payload) + log(f"nmem_exit={status}") + if output: + for line in output.splitlines(): + log(line) + log("") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/nowledge-mem-codex-plugin/scripts/install_hooks.py b/nowledge-mem-codex-plugin/scripts/install_hooks.py new file mode 100644 index 00000000..ead89b19 --- /dev/null +++ b/nowledge-mem-codex-plugin/scripts/install_hooks.py @@ -0,0 +1,205 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import importlib.util +import json +import re +import shutil +import stat +import sys +import tomllib +from datetime import datetime, timezone +from pathlib import Path + + +PLUGIN_ROOT = Path(__file__).resolve().parent.parent +HOME = Path.home() +CODEX_DIR = HOME / ".codex" +HOOKS_DIR = CODEX_DIR / "hooks" +GLOBAL_HOOKS_FILE = CODEX_DIR / "hooks.json" +CONFIG_FILE = CODEX_DIR / "config.toml" +SOURCE_HOOK = PLUGIN_ROOT / "hooks" / "nmem-stop-save.py" +INSTALLED_HOOK = HOOKS_DIR / "nowledge-mem-stop-save.py" +CODEX_HOOKS_KEY_RE = re.compile(r"^\s*codex_hooks\s*=") + + +def backup_invalid_json(path: Path, *, reason: str) -> dict: + backup = path.with_name(f"{path.name}.{datetime.now(timezone.utc).strftime('%Y%m%d%H%M%S')}.bak") + shutil.move(path, backup) + print(f"warning: moved {reason} to {backup}", file=sys.stderr) + return {} + + +def load_json(path: Path) -> dict: + if not path.exists(): + return {} + try: + payload = json.loads(path.read_text(encoding="utf-8")) + except json.JSONDecodeError: + return backup_invalid_json(path, reason="malformed JSON") + if not isinstance(payload, dict): + return backup_invalid_json(path, reason="non-object JSON") + return payload + + +def ensure_nmem_cli_runtime_ready() -> None: + required_modules = ("nmem_cli", "nmem_cli.session_import") + missing = [name for name in required_modules if importlib.util.find_spec(name) is None] + if not missing: + return + missing_text = ", ".join(missing) + raise SystemExit( + "This installer must be run with a Python interpreter that can import " + f"{missing_text}. Install nmem-cli into that interpreter, then rerun " + "scripts/install_hooks.py with the same python3." + ) + + +def normalize_hooks_doc(hooks_doc: dict) -> tuple[dict, list[dict]]: + hooks = hooks_doc.get("hooks") + if not isinstance(hooks, dict): + hooks = {} + hooks_doc["hooks"] = hooks + + stop_hooks = hooks.get("Stop") + if not isinstance(stop_hooks, list): + stop_hooks = [] + + normalized_stop_hooks: list[dict] = [] + for entry in stop_hooks: + if not isinstance(entry, dict): + continue + normalized_entry = dict(entry) + if not isinstance(normalized_entry.get("hooks"), list): + normalized_entry["hooks"] = [] + normalized_stop_hooks.append(normalized_entry) + + hooks["Stop"] = normalized_stop_hooks + return hooks_doc, normalized_stop_hooks + + +def rewrite_hook_shebang(path: Path) -> None: + original = path.read_text(encoding="utf-8") + lines = original.splitlines() + desired = f"#!{sys.executable}" + if lines and lines[0].startswith("#!"): + lines[0] = desired + else: + lines.insert(0, desired) + updated = "\n".join(lines) + if original.endswith("\n"): + updated += "\n" + path.write_text(updated, encoding="utf-8") + +def save_json(path: Path, payload: dict) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(payload, ensure_ascii=False, indent=2, sort_keys=True) + "\n", encoding="utf-8") + + +def install_runtime_hook() -> None: + HOOKS_DIR.mkdir(parents=True, exist_ok=True) + shutil.copy2(SOURCE_HOOK, INSTALLED_HOOK) + rewrite_hook_shebang(INSTALLED_HOOK) + mode = INSTALLED_HOOK.stat().st_mode + INSTALLED_HOOK.chmod(mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH) + + +def merge_hooks_json() -> None: + if GLOBAL_HOOKS_FILE.exists(): + backup = GLOBAL_HOOKS_FILE.with_suffix(".json.bak") + if not backup.exists(): + shutil.copy2(GLOBAL_HOOKS_FILE, backup) + hooks_doc = load_json(GLOBAL_HOOKS_FILE) + else: + hooks_doc = {} + + hooks_doc, stop_hooks = normalize_hooks_doc(hooks_doc) + + desired = { + "matcher": ".*", + "hooks": [ + { + "type": "command", + "command": str(INSTALLED_HOOK), + } + ], + } + + replaced = False + for entry in stop_hooks: + for hook_index, hook in enumerate(entry["hooks"]): + if not isinstance(hook, dict): + continue + if hook.get("command") == str(INSTALLED_HOOK): + entry["hooks"][hook_index] = dict(desired["hooks"][0]) + replaced = True + break + if replaced: + break + + if not replaced: + stop_hooks.append(desired) + + save_json(GLOBAL_HOOKS_FILE, hooks_doc) + + +def ensure_codex_hooks_enabled() -> None: + CONFIG_FILE.parent.mkdir(parents=True, exist_ok=True) + text = CONFIG_FILE.read_text(encoding="utf-8") if CONFIG_FILE.exists() else "" + if text.strip(): + tomllib.loads(text) + + lines = text.splitlines() + section_header = "[features]" + target_key = "codex_hooks" + features_start = None + features_end = len(lines) + + for index, line in enumerate(lines): + if line.strip() == section_header: + features_start = index + break + + if features_start is None: + if lines and lines[-1] != "": + lines.append("") + lines.extend([section_header, "codex_hooks = true"]) + else: + for index in range(features_start + 1, len(lines)): + stripped = lines[index].strip() + if stripped.startswith("[") and stripped.endswith("]"): + features_end = index + break + + replaced = False + for index in range(features_start + 1, features_end): + stripped = lines[index].strip() + if CODEX_HOOKS_KEY_RE.match(stripped): + lines[index] = "codex_hooks = true" + replaced = True + break + + if not replaced: + lines.insert(features_end, "codex_hooks = true") + + updated = "\n".join(lines) + if updated and not updated.endswith("\n"): + updated += "\n" + CONFIG_FILE.write_text(updated, encoding="utf-8") + + +def main() -> int: + ensure_nmem_cli_runtime_ready() + install_runtime_hook() + merge_hooks_json() + ensure_codex_hooks_enabled() + + print("Installed Nowledge Mem Codex Stop hook") + print(f"- runtime hook: {INSTALLED_HOOK}") + print(f"- hooks config: {GLOBAL_HOOKS_FILE}") + print(f"- feature flag ensured in: {CONFIG_FILE}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/nowledge-mem-codex-plugin/scripts/refresh_thread_titles.py b/nowledge-mem-codex-plugin/scripts/refresh_thread_titles.py new file mode 100644 index 00000000..3b7fd0d4 --- /dev/null +++ b/nowledge-mem-codex-plugin/scripts/refresh_thread_titles.py @@ -0,0 +1,203 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import importlib.util +import json +from pathlib import Path +from urllib.parse import quote + +SESSIONS_ROOT = Path.home() / ".codex" / "sessions" +AGENTS_PREFIX = "# AGENTS.md instructions for " +GET_TIMEOUT_SECONDS = 10.0 +IMPORT_TIMEOUT_SECONDS = 120.0 +cli = None +parse_codex_session_streaming = None + + +def load_hook_module(): + hook_path = Path(__file__).resolve().parent.parent / "hooks" / "nmem-stop-save.py" + spec = importlib.util.spec_from_file_location("nmem_stop_save", hook_path) + if spec is None: + raise RuntimeError(f"Failed to create import spec for {hook_path}") + if spec.loader is None: + raise RuntimeError(f"Failed to load hook module from {hook_path}: missing loader") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def iter_codex_rollouts() -> list[Path]: + return sorted(SESSIONS_ROOT.rglob("rollout-*.jsonl")) + + +def ensure_nmem_modules() -> tuple[object, object]: + global cli, parse_codex_session_streaming + if cli is None or parse_codex_session_streaming is None: + from nmem_cli import cli as cli_module + from nmem_cli.session_import import parse_codex_session_streaming as parser + + cli = cli_module + parse_codex_session_streaming = parser + return cli, parse_codex_session_streaming + + +def get_thread(thread_id: str) -> dict | None: + cli_module, _ = ensure_nmem_modules() + return cli_module.api_get_optional( + f"/threads/{quote(thread_id, safe='')}", + timeout=GET_TIMEOUT_SECONDS, + ) + + +def build_thread_payload( + thread_id: str, + title: str, + messages: list[dict], + thread: dict | None = None, +) -> dict: + thread = thread or {} + normalized_messages = [] + for message in messages: + if not isinstance(message, dict): + continue + role = message.get("role") + if not role: + continue + normalized_messages.append({"role": role, "content": message.get("content", "")}) + + return { + "thread_id": thread_id, + "title": title, + "source": thread.get("source") or "codex", + "project": thread.get("project"), + "workspace": thread.get("workspace"), + "metadata": thread.get("metadata") or {}, + "messages": normalized_messages, + } + + +def refresh_thread( + thread_id: str, + title: str, + messages: list[dict], + dry_run: bool, + original_title: str | None = None, + original_messages: list[dict] | None = None, + thread: dict | None = None, +) -> None: + if dry_run: + print(f"DRY RUN refresh {thread_id} -> {title}", flush=True) + return + + encoded_thread_id = quote(thread_id, safe='') + replacement_payload = build_thread_payload(thread_id, title, messages, thread=thread) + cli.api_delete(f"/threads/{encoded_thread_id}") + try: + cli.api_post( + "/threads", + replacement_payload, + timeout=IMPORT_TIMEOUT_SECONDS, + ) + except Exception: + if original_title is not None and original_messages is not None: + cli.api_post( + "/threads", + build_thread_payload( + thread_id, + original_title, + original_messages, + thread=thread, + ), + timeout=IMPORT_TIMEOUT_SECONDS, + ) + raise + print(f"REFRESHED {thread_id} -> {title}", flush=True) + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--dry-run", action="store_true") + parser.add_argument("--all", action="store_true") + parser.add_argument("--thread-id") + args = parser.parse_args() + + hook_module = load_hook_module() + hook_module.configure_nmem_env() + _, parse_codex = ensure_nmem_modules() + checked = 0 + refreshed = 0 + errors = 0 + + for rollout_path in iter_codex_rollouts(): + try: + parsed = parse_codex(rollout_path, truncate_large_content=True) + thread_id = parsed["thread_id"] + except Exception as exc: + errors += 1 + print(f"ERROR parse {rollout_path}: {exc}", flush=True) + continue + + if args.thread_id and thread_id != args.thread_id: + continue + + try: + thread_payload = get_thread(thread_id) + except SystemExit as exc: + errors += 1 + print(f"ERROR fetch {thread_id}: exited {exc.code}", flush=True) + continue + + if not thread_payload: + continue + + checked += 1 + if checked % 25 == 0: + print(f"PROGRESS checked={checked} refreshed={refreshed} errors={errors}", flush=True) + + thread = thread_payload.get("thread", {}) + current_title = (thread.get("title") or "").strip() + desired_title = hook_module.derive_thread_title( + parsed, + parsed.get("workspace") or parsed.get("metadata", {}).get("cwd") or "", + ) + + should_refresh = args.all or current_title.startswith(AGENTS_PREFIX) + if not should_refresh or current_title == desired_title: + continue + + try: + refresh_thread( + thread_id, + desired_title, + parsed.get("messages", []), + args.dry_run, + original_title=current_title, + original_messages=thread_payload.get("messages", []), + thread=thread, + ) + refreshed += 1 + except SystemExit as exc: + errors += 1 + print(f"ERROR refresh {thread_id}: exited {exc.code}", flush=True) + except Exception as exc: + errors += 1 + print(f"ERROR refresh {thread_id}: {exc}", flush=True) + + print( + json.dumps( + { + "checked": checked, + "refreshed": refreshed, + "errors": errors, + "dry_run": args.dry_run, + }, + ensure_ascii=False, + ), + flush=True, + ) + return 1 if errors else 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/nowledge-mem-codex-plugin/scripts/validate-plugin.mjs b/nowledge-mem-codex-plugin/scripts/validate-plugin.mjs new file mode 100644 index 00000000..f386c72c --- /dev/null +++ b/nowledge-mem-codex-plugin/scripts/validate-plugin.mjs @@ -0,0 +1,166 @@ +#!/usr/bin/env node + +import { existsSync, readFileSync, statSync } from "node:fs"; +import path from "node:path"; +import process from "node:process"; +import { fileURLToPath } from "node:url"; + +const scriptDir = path.dirname(fileURLToPath(import.meta.url)); +const pluginRoot = path.resolve(scriptDir, ".."); +const repoRoot = path.resolve(pluginRoot, ".."); + +const fail = (message) => { + console.error(`FAIL: ${message}`); + process.exitCode = 1; +}; + +const ok = (message) => { + console.log(`OK: ${message}`); +}; + +const readTextIfPresent = (fullPath, label) => { + if (!existsSync(fullPath)) { + fail(`missing ${label}`); + return null; + } + try { + return readFileSync(fullPath, "utf8"); + } catch (error) { + fail(`failed to read ${label}: ${error.message}`); + return null; + } +}; + +const parseJsonIfPresent = (fullPath, label) => { + const text = readTextIfPresent(fullPath, label); + if (text === null) { + return null; + } + try { + return JSON.parse(text); + } catch (error) { + fail(`failed to parse ${label}: ${error.message}`); + return null; + } +}; + +const requireFile = (relativePath) => { + const fullPath = path.join(pluginRoot, relativePath); + if (!existsSync(fullPath)) { + fail(`missing ${relativePath}`); + return null; + } + const stats = statSync(fullPath); + if (!stats.isFile() || stats.size === 0) { + fail(`empty or non-file ${relativePath}`); + return null; + } + ok(relativePath); + return fullPath; +}; + +for (const file of [ + ".codex-plugin/plugin.json", + "README.md", + "CHANGELOG.md", + "RELEASING.md", + "hooks/nmem-stop-save.py", + "scripts/install_hooks.py", + "scripts/validate-plugin.mjs", + "skills/working-memory/SKILL.md", + "skills/search-memory/SKILL.md", + "skills/save-thread/SKILL.md", + "skills/distill-memory/SKILL.md", + "skills/status/SKILL.md", +]) { + requireFile(file); +} + +const manifest = parseJsonIfPresent( + path.join(pluginRoot, ".codex-plugin/plugin.json"), + ".codex-plugin/plugin.json", +); +if (manifest) { + if (manifest.name !== "nowledge-mem") { + fail(`unexpected plugin name: ${manifest.name}`); + } else { + ok("plugin manifest name"); + } + if (manifest.version !== "0.1.3") { + fail(`expected version 0.1.3, got ${manifest.version}`); + } else { + ok("plugin manifest version"); + } +} + +const changelog = readTextIfPresent(path.join(pluginRoot, "CHANGELOG.md"), "CHANGELOG.md"); +if (changelog !== null) { + if (!changelog.includes("## [0.1.3]")) { + fail("CHANGELOG must contain a 0.1.3 entry"); + } else { + ok("CHANGELOG version entry"); + } +} + +const readme = readTextIfPresent(path.join(pluginRoot, "README.md"), "README.md"); +if (readme !== null) { + for (const phrase of [ + "scripts/install_hooks.py", + "~/.codex/hooks.json", + "codex_hooks = true", + "host-level", + "nmem_cli", + ]) { + if (!readme.includes(phrase)) { + fail(`README must mention ${phrase}`); + } else { + ok(`README mentions ${phrase}`); + } + } +} + +const integrationsDoc = parseJsonIfPresent(path.join(repoRoot, "integrations.json"), "integrations.json"); +if (integrationsDoc) { + const codexEntry = integrationsDoc.integrations?.find((entry) => entry.id === "codex-cli"); + if (!codexEntry) { + fail("integrations.json missing codex-cli entry"); + } else { + ok("integrations.json codex-cli entry"); + if (codexEntry.version !== "0.1.3") { + fail(`integrations.json codex-cli version must be 0.1.3, got ${codexEntry.version}`); + } else { + ok("integrations.json codex-cli version"); + } + if (!codexEntry.install?.command?.includes("nowledge-mem-codex-plugin/.")) { + fail("integrations.json install.command must copy nowledge-mem-codex-plugin/. so hidden files are preserved"); + } else { + ok("integrations.json install.command copies hidden files"); + } + if (!codexEntry.install?.command?.includes("install_hooks.py")) { + fail("integrations.json install.command must run scripts/install_hooks.py so Codex auto-capture is actually enabled"); + } else { + ok("integrations.json install.command runs install_hooks.py"); + } + if (!codexEntry.install?.updateCommand?.includes("nowledge-mem-codex-plugin/.")) { + fail("integrations.json install.updateCommand must copy nowledge-mem-codex-plugin/. so hidden files are preserved"); + } else { + ok("integrations.json updateCommand copies hidden files"); + } + if (!codexEntry.install?.updateCommand?.includes("install_hooks.py")) { + fail("integrations.json install.updateCommand must rerun scripts/install_hooks.py so hook config stays current"); + } else { + ok("integrations.json updateCommand runs install_hooks.py"); + } + if (codexEntry.capabilities?.autoCapture !== true) { + fail(`integrations.json codex-cli autoCapture must be true, got ${codexEntry.capabilities?.autoCapture}`); + } else { + ok("integrations.json codex-cli autoCapture"); + } + } +} + +if (process.exitCode) { + process.exit(process.exitCode); +} + +console.log("Codex plugin validation passed."); diff --git a/nowledge-mem-codex-plugin/tests/test_codex_plugin.py b/nowledge-mem-codex-plugin/tests/test_codex_plugin.py new file mode 100644 index 00000000..c3a4d24e --- /dev/null +++ b/nowledge-mem-codex-plugin/tests/test_codex_plugin.py @@ -0,0 +1,619 @@ +import importlib.util +import tempfile +import unittest +from pathlib import Path +from types import ModuleType +from unittest import mock + + +PLUGIN_ROOT = Path(__file__).resolve().parent.parent +HOOK_MODULE_PATH = PLUGIN_ROOT / "hooks" / "nmem-stop-save.py" +INSTALL_MODULE_PATH = PLUGIN_ROOT / "scripts" / "install_hooks.py" +REFRESH_MODULE_PATH = PLUGIN_ROOT / "scripts" / "refresh_thread_titles.py" + + +def load_module(module_name: str, module_path: Path): + spec = importlib.util.spec_from_file_location(module_name, module_path) + module = importlib.util.module_from_spec(spec) + assert spec.loader is not None + spec.loader.exec_module(module) + return module + + +class HookTests(unittest.TestCase): + def setUp(self): + self.module = load_module("nmem_stop_save", HOOK_MODULE_PATH) + self.temp_dir = tempfile.TemporaryDirectory() + temp_path = Path(self.temp_dir.name) + self.module.LOG_FILE = temp_path / "nmem-stop-hook.log" + self.module.STATE_FILE = temp_path / "hook-state.json" + + def tearDown(self): + self.temp_dir.cleanup() + + def patch_nmem_modules(self, *, parser, api_get_optional=None, api_post=None): + package = ModuleType("nmem_cli") + cli_module = ModuleType("nmem_cli.cli") + cli_module.api_get_optional = api_get_optional or mock.Mock(return_value=None) + cli_module.api_post = api_post or mock.Mock() + session_module = ModuleType("nmem_cli.session_import") + session_module.parse_codex_session_streaming = parser + return mock.patch.dict( + "sys.modules", + { + "nmem_cli": package, + "nmem_cli.cli": cli_module, + "nmem_cli.session_import": session_module, + }, + ) + + def test_missing_transcript_path_skips_before_path_construction(self): + status, output = self.module.import_current_transcript( + { + "session_id": "session-1", + "cwd": "/tmp/project", + "hook_event_name": "Stop", + } + ) + + self.assertEqual(status, 0) + self.assertIn("transcript_path missing or unreadable", output) + + def test_directory_transcript_path_is_rejected(self): + temp_path = Path(self.temp_dir.name) + + status, output = self.module.import_current_transcript( + { + "session_id": "session-2", + "cwd": "/tmp/project", + "hook_event_name": "Stop", + "transcript_path": str(temp_path), + } + ) + + self.assertEqual(status, 0) + self.assertIn("transcript_path missing or unreadable", output) + + def test_api_sync_failure_is_reported_and_state_not_written(self): + transcript_path = Path(self.temp_dir.name) / "rollout.jsonl" + transcript_path.write_text("placeholder", encoding="utf-8") + + parsed = { + "thread_id": "codex-error-thread", + "title": "Error thread", + "messages": [{"role": "user", "content": "hi"}], + } + + parser_mock = mock.Mock(return_value=parsed) + api_post = mock.Mock(side_effect=RuntimeError("boom")) + with self.patch_nmem_modules( + parser=parser_mock, + api_get_optional=mock.Mock(return_value=None), + api_post=api_post, + ): + status, output = self.module.import_current_transcript( + { + "session_id": "session-3", + "cwd": "/tmp/project", + "hook_event_name": "Stop", + "transcript_path": str(transcript_path), + } + ) + + self.assertEqual(status, 0) + self.assertIn("failed to sync", output) + self.assertFalse(self.module.STATE_FILE.exists()) + + def test_load_json_returns_empty_dict_for_non_object_json(self): + payload_path = Path(self.temp_dir.name) / "payload.json" + payload_path.write_text('["not", "a", "dict"]', encoding="utf-8") + + payload = self.module.load_json(payload_path) + + self.assertEqual(payload, {}) + + def test_subsequent_import_only_sends_delta_messages(self): + transcript_path = Path(self.temp_dir.name) / "rollout.jsonl" + transcript_path.write_text("placeholder", encoding="utf-8") + + parsed_first = { + "thread_id": "codex-delta-thread", + "title": "Delta thread", + "messages": [ + {"role": "user", "content": "u1"}, + {"role": "assistant", "content": "a1"}, + ], + } + parsed_second = { + "thread_id": "codex-delta-thread", + "title": "Delta thread", + "messages": [ + {"role": "user", "content": "u1"}, + {"role": "assistant", "content": "a1"}, + {"role": "user", "content": "u2"}, + ], + } + + parser_mock = mock.Mock(side_effect=[parsed_first, parsed_second]) + api_get_optional = mock.Mock(side_effect=[None, {"thread": {"thread_id": "codex-delta-thread"}}]) + api_post = mock.Mock() + with self.patch_nmem_modules( + parser=parser_mock, + api_get_optional=api_get_optional, + api_post=api_post, + ): + first_status, _ = self.module.import_current_transcript( + { + "session_id": "session-delta", + "cwd": "/tmp/project", + "hook_event_name": "Stop", + "transcript_path": str(transcript_path), + } + ) + second_status, _ = self.module.import_current_transcript( + { + "session_id": "session-delta", + "cwd": "/tmp/project", + "hook_event_name": "Stop", + "transcript_path": str(transcript_path), + } + ) + + self.assertEqual(first_status, 0) + self.assertEqual(second_status, 0) + self.assertEqual(api_post.call_count, 2) + + create_endpoint, create_payload = api_post.call_args_list[0].args + self.assertEqual(create_endpoint, "/threads") + self.assertEqual(len(create_payload["messages"]), 2) + self.assertEqual(create_payload["messages"][0]["content"], "u1") + + append_endpoint, append_payload = api_post.call_args_list[1].args + self.assertEqual(append_endpoint, "/threads/codex-delta-thread/append") + self.assertTrue(append_payload["deduplicate"]) + self.assertEqual(len(append_payload["messages"]), 1) + self.assertEqual(append_payload["messages"][0]["content"], "u2") + + def test_malformed_messages_are_filtered_before_sync(self): + transcript_path = Path(self.temp_dir.name) / "rollout.jsonl" + transcript_path.write_text("placeholder", encoding="utf-8") + + parsed = { + "thread_id": "codex-filter-thread", + "title": "Filter thread", + "messages": [ + {"role": "user", "content": "u1"}, + {"content": "missing-role"}, + "bad-entry", + {"role": "assistant", "content": "a1"}, + ], + } + + api_post = mock.Mock() + with self.patch_nmem_modules( + parser=mock.Mock(return_value=parsed), + api_get_optional=mock.Mock(return_value=None), + api_post=api_post, + ): + status, _ = self.module.import_current_transcript( + { + "session_id": "session-filter", + "cwd": "/tmp/project", + "hook_event_name": "Stop", + "transcript_path": str(transcript_path), + } + ) + + self.assertEqual(status, 0) + _, payload = api_post.call_args.args + self.assertEqual( + payload["messages"], + [ + {"role": "user", "content": "u1"}, + {"role": "assistant", "content": "a1"}, + ], + ) + + def test_state_lock_times_out_when_contended(self): + with mock.patch.object(self.module.fcntl, "flock", side_effect=BlockingIOError), \ + mock.patch.object( + self.module.time, + "monotonic", + side_effect=[0.0, 0.0, 0.2, 0.4], + ), \ + mock.patch.object(self.module.time, "sleep"): + with self.assertRaises(TimeoutError): + with self.module.state_lock(timeout_seconds=0.3): + self.fail("lock should not be acquired") + + def test_missing_remote_thread_recreates_full_transcript(self): + transcript_path = Path(self.temp_dir.name) / "rollout.jsonl" + transcript_path.write_text("placeholder", encoding="utf-8") + + parsed = { + "thread_id": "codex-recreate-thread", + "title": "Recreate thread", + "messages": [ + {"role": "user", "content": "u1"}, + {"role": "assistant", "content": "a1"}, + {"role": "user", "content": "u2"}, + ], + } + + state = { + "session-recreate": { + "thread_id": "codex-recreate-thread", + "message_count": 2, + "source_file": str(transcript_path), + } + } + self.module.save_json(self.module.STATE_FILE, state) + + api_post = mock.Mock() + with self.patch_nmem_modules( + parser=mock.Mock(return_value=parsed), + api_get_optional=mock.Mock(return_value=None), + api_post=api_post, + ): + status, _ = self.module.import_current_transcript( + { + "session_id": "session-recreate", + "cwd": "/tmp/project", + "hook_event_name": "Stop", + "transcript_path": str(transcript_path), + } + ) + + self.assertEqual(status, 0) + endpoint, payload = api_post.call_args.args + self.assertEqual(endpoint, "/threads") + self.assertEqual(len(payload["messages"]), 3) + + def test_title_skips_agents_preamble_and_uses_first_real_user_message(self): + parsed = { + "title": "# AGENTS.md instructions for /Users/hansonmei/Projects/nowledge-community", + "messages": [ + {"role": "developer", "content": "..."}, + { + "role": "user", + "content": "# AGENTS.md instructions for /Users/hansonmei/Projects/nowledge-community\n\n...", + }, + { + "role": "user", + "content": "codex://threads/019d7145-eb73-7840-84e5-bef5c0f19261，你看看这个对话里的修改是不是已经落盘到本目录了", + }, + ], + } + + title = self.module.derive_thread_title(parsed, "/Users/hansonmei/Projects/nowledge-community") + + self.assertEqual( + title, + "这个对话里的修改是否已经落盘到本目录了", + ) + + def test_title_falls_back_to_workspace_name_when_only_agents_preamble_exists(self): + parsed = { + "title": "# AGENTS.md instructions for /Users/hansonmei/Projects/nowledge-community", + "messages": [ + { + "role": "user", + "content": "# AGENTS.md instructions for /Users/hansonmei/Projects/nowledge-community\n\n...", + } + ], + } + + title = self.module.derive_thread_title(parsed, "/Users/hansonmei/Projects/nowledge-community") + + self.assertEqual(title, "Codex: nowledge-community") + + def test_title_skips_environment_context_blocks(self): + parsed = { + "title": "# AGENTS.md instructions for /Users/hansonmei", + "messages": [ + {"role": "user", "content": "# AGENTS.md instructions for /Users/hansonmei\n\n..."}, + { + "role": "user", + "content": "\n /Users/hansonmei\n", + }, + {"role": "user", "content": "更新我的openclaw"}, + ], + } + + title = self.module.derive_thread_title(parsed, "/Users/hansonmei") + + self.assertEqual(title, "更新我的openclaw") + + def test_title_skips_generic_agents_document_blocks(self): + parsed = { + "title": "# AGENTS.md", + "messages": [ + { + "role": "user", + "content": ( + "# AGENTS.md\n\n" + "## Purpose\n\n" + "only applicable in codex app\n\n" + "\n" + "Use nmem.\n" + "" + ), + }, + {"role": "user", "content": "codex app 支持hook吗"}, + ], + } + + title = self.module.derive_thread_title(parsed, "/Users/hansonmei") + + self.assertEqual(title, "codex app 支持hook吗") + + def test_title_extracts_request_from_files_preamble(self): + parsed = { + "title": "# AGENTS.md instructions for /Users/hansonmei", + "messages": [ + {"role": "user", "content": "# AGENTS.md instructions for /Users/hansonmei\n\n..."}, + { + "role": "user", + "content": ( + "# Files mentioned by the user:\n\n" + "## a.png: /tmp/a.png\n\n" + "## My request for Codex:\n" + "这三张截图都要提取文字，必要时可以先切分再提取" + ), + }, + ], + } + + title = self.module.derive_thread_title(parsed, "/Users/hansonmei") + + self.assertEqual(title, "这三张截图都要提取文字，必要时可以先切分再提取") + + +class InstallHookTests(unittest.TestCase): + def setUp(self): + self.module = load_module("install_hooks", INSTALL_MODULE_PATH) + self.temp_dir = tempfile.TemporaryDirectory() + temp_path = Path(self.temp_dir.name) + self.module.CODEX_DIR = temp_path / ".codex" + self.module.HOOKS_DIR = self.module.CODEX_DIR / "hooks" + self.module.GLOBAL_HOOKS_FILE = self.module.CODEX_DIR / "hooks.json" + self.module.CONFIG_FILE = self.module.CODEX_DIR / "config.toml" + self.module.INSTALLED_HOOK = self.module.HOOKS_DIR / "nowledge-mem-stop-save.py" + + def tearDown(self): + self.temp_dir.cleanup() + + def test_load_json_recovers_from_malformed_json(self): + self.module.GLOBAL_HOOKS_FILE.parent.mkdir(parents=True, exist_ok=True) + self.module.GLOBAL_HOOKS_FILE.write_text("{not-json", encoding="utf-8") + + payload = self.module.load_json(self.module.GLOBAL_HOOKS_FILE) + + self.assertEqual(payload, {}) + backups = list(self.module.GLOBAL_HOOKS_FILE.parent.glob("hooks.json.*.bak")) + self.assertEqual(len(backups), 1) + + def test_load_json_recovers_from_non_object_json(self): + self.module.GLOBAL_HOOKS_FILE.parent.mkdir(parents=True, exist_ok=True) + self.module.GLOBAL_HOOKS_FILE.write_text('["not-an-object"]', encoding="utf-8") + + payload = self.module.load_json(self.module.GLOBAL_HOOKS_FILE) + + self.assertEqual(payload, {}) + backups = list(self.module.GLOBAL_HOOKS_FILE.parent.glob("hooks.json.*.bak")) + self.assertEqual(len(backups), 1) + + def test_merge_hooks_json_normalizes_malformed_stop_section(self): + self.module.GLOBAL_HOOKS_FILE.parent.mkdir(parents=True, exist_ok=True) + self.module.GLOBAL_HOOKS_FILE.write_text( + '{"hooks": {"Stop": {"matcher": ".*"}}}', + encoding="utf-8", + ) + + self.module.merge_hooks_json() + payload = self.module.load_json(self.module.GLOBAL_HOOKS_FILE) + + self.assertIsInstance(payload["hooks"]["Stop"], list) + self.assertEqual(len(payload["hooks"]["Stop"]), 1) + self.assertEqual( + payload["hooks"]["Stop"][0]["hooks"][0]["command"], + str(self.module.INSTALLED_HOOK), + ) + + def test_merge_hooks_json_preserves_existing_stop_entry_shape(self): + self.module.GLOBAL_HOOKS_FILE.parent.mkdir(parents=True, exist_ok=True) + payload = { + "hooks": { + "Stop": [ + { + "matcher": "workspace-1", + "hooks": [ + {"type": "command", "command": str(self.module.INSTALLED_HOOK)}, + {"type": "command", "command": "/usr/bin/other-hook"}, + ], + } + ] + } + } + self.module.save_json(self.module.GLOBAL_HOOKS_FILE, payload) + + self.module.merge_hooks_json() + updated = self.module.load_json(self.module.GLOBAL_HOOKS_FILE) + + stop_entry = updated["hooks"]["Stop"][0] + self.assertEqual(stop_entry["matcher"], "workspace-1") + self.assertEqual(len(stop_entry["hooks"]), 2) + self.assertEqual(stop_entry["hooks"][1]["command"], "/usr/bin/other-hook") + + def test_merge_hooks_json_copies_backup_before_invalid_payload_is_rotated(self): + self.module.GLOBAL_HOOKS_FILE.parent.mkdir(parents=True, exist_ok=True) + self.module.GLOBAL_HOOKS_FILE.write_text("{not-json", encoding="utf-8") + + self.module.merge_hooks_json() + + self.assertTrue(self.module.GLOBAL_HOOKS_FILE.with_suffix(".json.bak").exists()) + rotated = list(self.module.GLOBAL_HOOKS_FILE.parent.glob("hooks.json.*.bak")) + self.assertEqual(len(rotated), 1) + + def test_ensure_codex_hooks_enabled_only_changes_features_section(self): + self.module.CONFIG_FILE.parent.mkdir(parents=True, exist_ok=True) + self.module.CONFIG_FILE.write_text( + "\n".join( + [ + '[projects."/tmp/demo"]', + "codex_hooks = false", + "", + "[features]", + "apps = true", + "codex_hooks = false", + "", + ] + ), + encoding="utf-8", + ) + + self.module.ensure_codex_hooks_enabled() + updated = self.module.CONFIG_FILE.read_text(encoding="utf-8") + + self.assertIn('[projects."/tmp/demo"]\ncodex_hooks = false', updated) + self.assertIn("[features]\napps = true\ncodex_hooks = true", updated) + + def test_ensure_codex_hooks_enabled_replaces_indented_key_without_duplication(self): + self.module.CONFIG_FILE.parent.mkdir(parents=True, exist_ok=True) + self.module.CONFIG_FILE.write_text( + "\n".join( + [ + "[features]", + " codex_hooks = false", + "apps = true", + "", + ] + ), + encoding="utf-8", + ) + + self.module.ensure_codex_hooks_enabled() + updated = self.module.CONFIG_FILE.read_text(encoding="utf-8") + + self.assertEqual(updated.count("codex_hooks = true"), 1) + self.assertNotIn("codex_hooks = false", updated) + self.assertIn("[features]\ncodex_hooks = true\napps = true\n", updated) + + def test_install_runtime_hook_rewrites_shebang_to_current_python(self): + self.module.install_runtime_hook() + + installed = self.module.INSTALLED_HOOK.read_text(encoding="utf-8") + self.assertTrue(installed.startswith(f"#!{self.module.sys.executable}\n")) + self.assertNotEqual(self.module.INSTALLED_HOOK.stat().st_mode & 0o111, 0) + + def test_ensure_nmem_cli_runtime_ready_exits_when_missing(self): + with mock.patch.object(self.module.importlib.util, "find_spec", return_value=None): + with self.assertRaises(SystemExit) as exc: + self.module.ensure_nmem_cli_runtime_ready() + + self.assertIn("nmem_cli", str(exc.exception)) + + +class RefreshThreadTitleTests(unittest.TestCase): + def setUp(self): + self.module = load_module("refresh_thread_titles", REFRESH_MODULE_PATH) + + def test_refresh_thread_restores_original_messages_on_failure(self): + cli_mock = mock.Mock() + cli_mock.api_post.side_effect = [RuntimeError("create failed"), None] + + with mock.patch.object(self.module, "cli", cli_mock): + with self.assertRaises(RuntimeError): + self.module.refresh_thread( + thread_id="codex-thread-1", + title="New title", + messages=[{"role": "user", "content": "new"}], + dry_run=False, + original_title="Old title", + original_messages=[{"role": "user", "content": "old"}], + ) + + self.assertEqual(cli_mock.api_delete.call_count, 1) + self.assertEqual(cli_mock.api_post.call_count, 2) + + first_endpoint, first_payload = cli_mock.api_post.call_args_list[0].args + self.assertEqual(first_endpoint, "/threads") + self.assertEqual(first_payload["title"], "New title") + + restore_endpoint, restore_payload = cli_mock.api_post.call_args_list[1].args + self.assertEqual(restore_endpoint, "/threads") + self.assertEqual(restore_payload["title"], "Old title") + self.assertEqual( + restore_payload["messages"], + [{"role": "user", "content": "old"}], + ) + + def test_build_thread_payload_filters_malformed_messages(self): + payload = self.module.build_thread_payload( + "thread-1", + "Title", + [ + {"role": "user", "content": "ok"}, + {"content": "missing-role"}, + "bad-entry", + {"role": "assistant"}, + ], + ) + + self.assertEqual( + payload["messages"], + [ + {"role": "user", "content": "ok"}, + {"role": "assistant", "content": ""}, + ], + ) + + def test_main_configures_nmem_env_before_importing_modules(self): + hook_module = mock.Mock() + hook_module.derive_thread_title.return_value = "ignored" + temp_dir = tempfile.TemporaryDirectory() + self.addCleanup(temp_dir.cleanup) + + with mock.patch.object(self.module, "load_hook_module", return_value=hook_module), \ + mock.patch.object(self.module, "ensure_nmem_modules", return_value=(mock.Mock(), mock.Mock())), \ + mock.patch.object(self.module, "iter_codex_rollouts", return_value=[]), \ + mock.patch("sys.argv", ["refresh_thread_titles.py"]): + result = self.module.main() + + self.assertEqual(result, 0) + hook_module.configure_nmem_env.assert_called_once_with() + + def test_main_counts_generic_refresh_errors(self): + temp_dir = tempfile.TemporaryDirectory() + self.addCleanup(temp_dir.cleanup) + rollout_path = Path(temp_dir.name) / "rollout-1.jsonl" + rollout_path.write_text("placeholder", encoding="utf-8") + + self.module.parse_codex_session_streaming = mock.Mock( + return_value={ + "thread_id": "codex-thread-1", + "messages": [{"role": "user", "content": "real request"}], + "workspace": "/tmp/project", + } + ) + self.module.cli = mock.Mock() + + with mock.patch.object(self.module, "iter_codex_rollouts", return_value=[rollout_path]), \ + mock.patch.object( + self.module, + "get_thread", + return_value={"thread": {"title": f"{self.module.AGENTS_PREFIX}/tmp/project"}}, + ), \ + mock.patch.object(self.module, "refresh_thread", side_effect=RuntimeError("boom")), \ + mock.patch("sys.argv", ["refresh_thread_titles.py"]), \ + mock.patch("builtins.print") as print_mock: + result = self.module.main() + + self.assertNotEqual(result, 0) + printed_lines = [" ".join(str(arg) for arg in call.args) for call in print_mock.call_args_list] + self.assertTrue(any("ERROR refresh codex-thread-1: boom" in line for line in printed_lines)) + self.assertTrue(any('"errors": 1' in line for line in printed_lines)) + + +if __name__ == "__main__": + unittest.main() diff --git a/nowledge-mem-codex-prompts/README.md b/nowledge-mem-codex-prompts/README.md index 5d8e18cf..97414435 100644 --- a/nowledge-mem-codex-prompts/README.md +++ b/nowledge-mem-codex-prompts/README.md @@ -1,6 +1,6 @@ # Nowledge Mem for Codex CLI -> **Deprecated**: This custom prompts package has been superseded by the **[Codex Plugin](../nowledge-mem-codex-plugin/)**. The plugin ships the same Codex-facing memory behaviors as packaged skills with a cleaner install/update path. See the [migration guide](https://mem.nowledge.co/docs/integrations/codex-cli#migrating-from-custom-prompts) for details. +> **Deprecated**: This custom prompts package has been superseded by the **[Codex Plugin](../nowledge-mem-codex-plugin/)**. The plugin ships the same Codex-facing memory behaviors as packaged skills with a cleaner install/update path, plus an optional host-level hook installer for automatic transcript capture after you run it. See the [migration guide](https://mem.nowledge.co/docs/integrations/codex-cli#migrating-from-custom-prompts) for details. > Memory-aware custom prompts for Codex CLI, with an optional project `AGENTS.md` companion for stronger default behavior. diff --git a/nowledge-mem-npx-skills/skills/check-integration/SKILL.md b/nowledge-mem-npx-skills/skills/check-integration/SKILL.md index 4ed49336..a8db3d8d 100644 --- a/nowledge-mem-npx-skills/skills/check-integration/SKILL.md +++ b/nowledge-mem-npx-skills/skills/check-integration/SKILL.md @@ -61,7 +61,7 @@ The canonical source for this table is `community/integrations.json`. | **Gemini CLI** | Running as Gemini CLI agent; `~/.gemini/` exists | Search "Nowledge Mem" in the Gemini CLI Extensions Gallery | [Guide](https://mem.nowledge.co/docs/integrations/gemini-cli) | | **Alma** | Running inside Alma; `~/.config/alma/` exists | In Alma: Settings > Plugins > Marketplace, search "Nowledge Mem" | [Guide](https://mem.nowledge.co/docs/integrations/alma) | | **Droid** | Running inside Droid (Factory) | Add nowledge-co/community marketplace, install nowledge-mem@nowledge-community | [Guide](https://mem.nowledge.co/docs/integrations/droid) | -| **Codex CLI** | Running as Codex CLI agent; `~/.codex/` exists | Copy `nowledge-mem-codex-plugin` to `~/.codex/plugins/cache/local/nowledge-mem/local/`, then enable `[features] plugins = true` and `[plugins."nowledge-mem@local"] enabled = true` in `~/.codex/config.toml` | [Guide](https://mem.nowledge.co/docs/integrations/codex-cli) | +| **Codex CLI** | Running as Codex CLI agent; `~/.codex/` exists | Copy `nowledge-mem-codex-plugin` to `~/.codex/plugins/cache/local/nowledge-mem/local/`, enable `[features] plugins = true` and `[plugins."nowledge-mem@local"] enabled = true` in `~/.codex/config.toml`, then run `python3 ~/.codex/plugins/cache/local/nowledge-mem/local/scripts/install_hooks.py` to enable automatic capture | [Guide](https://mem.nowledge.co/docs/integrations/codex-cli) | | **Bub** | Running inside Bub | `pip install nowledge-mem-bub` | [Guide](https://mem.nowledge.co/docs/integrations/bub) | | **Pi** | Running as Pi agent; `~/.pi/` exists | `pi install npm:nowledge-mem-pi` | [Guide](https://mem.nowledge.co/docs/integrations/pi) | | **OpenCode** | Running as OpenCode agent; `~/.config/opencode/` or `.opencode/` exists | Add `"opencode-nowledge-mem"` to `opencode.json` plugin array | [Guide](https://mem.nowledge.co/docs/integrations/opencode) |