MCP Bridge (ctxce) + Optional Auth: single MCP server (HTTP/stdio) for Indexer & Memory, VS Code auto‑wiring, backend auth endpoints

[voarsh2]

Summary

This PR introduces a consolidated MCP bridge CLI (“ctxce”) and opt‑in backend authentication for the Context Engine stack. The bridge exposes a single MCP server over HTTP or stdio that proxies to the Indexer and Memory services, centralizes per‑session defaults, and adds resiliency. The VS Code extension gains auto‑wiring for the HTTP bridge, unified MCP config writing, and a guided sign‑in flow. Backend services provide lightweight auth endpoints (SQLite‑backed) with optional collection ACL enforcement. Behavior remains unchanged unless auth or ACL is enabled.

Motivation

• Reduce IDE configuration friction: one MCP server instead of separate indexer/memory entries and transports.
• Centralize session defaults (collection, mode, under, language) to avoid repetitive knob passing in agent workflows.
• Provide a simple, opt‑in auth model for shared deployments (session TTLs, token/password login) ahead of org/user features.
• Improve reliability of tool calls under transient errors; align extension workflows with the bridge.

What’s Included

1) MCP bridge CLI (Node 18+)

• New package: ctx-mcp-bridge/ published as @context-engine-bridge/context-engine-mcp-bridge.
• Commands:
  • ctxce mcp-serve – stdio MCP bridge.
  • ctxce mcp-http-serve – HTTP MCP bridge (default http://127.0.0.1:<port>/mcp).
  • ctxce auth <login|status|logout> – helper commands for token/password flows.
• Bridge features:
  • Proxies tools/list and tools/call; injects set_session_defaults (session, collection, mode, under, language).
  • Resiliency: configurable timeouts, transient‑error retries, and session re‑initialization when remote sessions expire.
  • Optional debug logging (CTXCE_DEBUG_LOG); exit when stdin closes (CTXCE_EXIT_ON_STDIN_CLOSE=1).
  • Path remapping for search tool results to workspace‑relative paths (toggle via CTXCE_BRIDGE_MAP_PATHS).

2) Optional backend auth (OFF by default)

• New module: scripts/auth_backend.py (SQLite users/sessions, PBKDF2 password hashing, sliding TTL).
• Upload service endpoints:
  • GET /auth/status – enabled, has_users, TTL.
  • POST /auth/login – token flow (client/workspace), gated by CTXCE_AUTH_SHARED_TOKEN or CTXCE_AUTH_ALLOW_OPEN_TOKEN_LOGIN=1.
  • POST /auth/login/password – username/password flow.
  • POST /auth/users – bootstrap first user; subsequent users require X-Admin-Token (from CTXCE_AUTH_ADMIN_TOKEN).
• Admin UI (served by upload service):
  • /admin with bootstrap, login, ACL pages.
  • Grants/revokes collection access; sets cookie ctxce_session for admin flows.
• Auth registry integration:
  • Health check syncs Qdrant collections into the auth DB (if enabled).
  • Optional ACL enforcement toggles (CTXCE_MCP_ACL_ENFORCE, CTXCE_ACL_ALLOW_ALL).

3) Indexer/Memory MCP auth enforcement + session defaults

• Indexer (scripts/mcp_indexer_server.py):
  • Optional session enforcement on protected tools when auth is enabled.
  • set_session_defaults supports collection, mode, under, language; empty strings explicitly unset sticky defaults.
  • repo_search resolves defaults from per‑connection/session scopes, token defaults, and env fallback.
  • Tolerates bridges that omit the MCP session header during handshake.
• Memory (scripts/mcp_memory_server.py):
  • Optional session enforcement on memory.* tools (find, store) when auth is enabled.
  • HNSW index parameters set on collection creation for better recall at scale.
  • Fastembed model cache and lazy warmups reduce cold‑start latency.

4) Upload clients: session propagation + error handling

• scripts/remote_upload_client.py and scripts/standalone_upload_client.py:
  • Include session id with uploads (resolved from env or ~/.ctxce/auth.json).
  • Stat caches avoid re‑hashing unchanged files during --watch.
  • SimpleHashCache fixes reduce O(N²) cache churn on Windows watch runs.
  • Explicit 401 handling: returns UNAUTHORIZED with guidance (“log in with ctxce auth login”).

5) Upload service hardening

• Guard against nested dev‑workspace slugs to prevent recursive workspace nesting.
• Enforce collection write ACLs on uploads when CTXCE_MCP_ACL_ENFORCE=1 and auth is enabled.
• Initialize .codebase permissions and group for shared volumes.
• Path Traversal hardening in upload_service

6) VS Code extension integration

• Settings:
  • contextEngineUploader.mcpServerMode: bridge | direct.
  • contextEngineUploader.mcpTransportMode: sse-remote | http.
  • contextEngineUploader.autoStartMcpBridge: auto‑launch ctxce mcp-http-serve.
  • contextEngineUploader.mcpBridgePort, mcpBridgeBinPath, mcpBridgeLocalOnly.
• MCP config writer:
  • bridge-stdio: single stdio MCP server via ctxce mcp-serve (behind SSE).
  • bridge-http: single HTTP MCP server via ctxce mcp-http-serve (auto‑start).
  • direct-sse/direct-http: two servers (indexer, memory) without bridge.
• Guided sign‑in using ctxce auth commands; MCP config refresh/hot‑reload.

Configuration

Bridge CLI (env)

• CTXCE_INDEXER_URL – MCP indexer URL (default http://localhost:8003/mcp).
• CTXCE_MEMORY_URL – MCP memory URL (optional, default http://localhost:8002/mcp).
• CTXCE_HTTP_PORT – HTTP bridge port (default 30810).
• Resiliency:
  • CTXCE_TOOL_TIMEOUT_MSEC – per‑tool call timeout (default 300000).
  • CTXCE_TOOL_RETRY_ATTEMPTS – retry attempts (default 2).
  • CTXCE_TOOL_RETRY_DELAY_MSEC – delay between retries (default 200).
• Logging/exit:
  • CTXCE_DEBUG_LOG – write bridge logs to file.
  • CTXCE_EXIT_ON_STDIN_CLOSE=1 – exit on stdio close.

Backend auth (env)

• CTXCE_AUTH_ENABLED=1 – enable auth (default OFF).
• CTXCE_AUTH_SHARED_TOKEN – shared token for /auth/login (token flow).
• CTXCE_AUTH_ALLOW_OPEN_TOKEN_LOGIN – allow token login without shared token (dev only).
• CTXCE_AUTH_ADMIN_TOKEN – required for /auth/users after bootstrap.
• CTXCE_AUTH_DB_URL – SQLite path (default /work/.codebase/ctxce_auth.sqlite).
• CTXCE_AUTH_SESSION_TTL_SECONDS – session TTL; <=0 disables expiry (default 0).
• Collection ACLs:
  • CTXCE_ACL_ALLOW_ALL – bypass ACLs.
  • CTXCE_MCP_ACL_ENFORCE – enforce collection ACLs in MCP servers.
• Bridge auth CLI:
  • CTXCE_AUTH_BACKEND_URL – upload service base URL.

Extension (typical)

{
  "contextEngineUploader.mcpServerMode": "bridge",
  "contextEngineUploader.mcpTransportMode": "http",
  "contextEngineUploader.autoStartMcpBridge": true,
  "contextEngineUploader.mcpBridgePort": 30810
}

[voarsh2]

I am now evaluating the approach - over making it a http service, separate from local stdio - since I encountered annoyances in how IDE clients manage stdio processes.......

[voarsh2]

Doing last minute test fix (missing dep in test workflow) and a quick check then ready for merge.