Skip to content

Project-scoped search sidecars (one per project, not per instance)#63

Merged
elliottregan merged 7 commits intomainfrom
feat/project-scoped-sidecars
Apr 22, 2026
Merged

Project-scoped search sidecars (one per project, not per instance)#63
elliottregan merged 7 commits intomainfrom
feat/project-scoped-sidecars

Conversation

@elliottregan
Copy link
Copy Markdown
Owner

@elliottregan elliottregan commented Apr 22, 2026

Summary

Closes #62

  • Hoists the five search sidecars (qdrant, llama-server, llama-clustering, reduce-api, hdbscan-api) from per-instance scope into per-project scope, so every instance in a project shares one set of sidecars instead of spinning up its own copy.
  • Adds docker-compose.project.yml — a new compose template declaring the five sidecars at project scope, with per-project named volumes (cspace-<project>-qdrant, cspace-<project>-llama-models) and Docker network aliases on the project network.
  • Adds cspace down --project flag to tear down all instances AND the shared search stack for the current project.

Architecture

project network: cspace-<project>-net

  project stack (N=1, compose: cspace-<project>-stack)
    qdrant, llama-server, llama-clustering,
    reduce-api, hdbscan-api

  instance stacks (N: mercury, venus, ...)
    cspace service + browser sidecar + search-mcp

Each instance resolves sidecars via network aliases (qdrant, llama-server, etc.) — the sidecar URLs in search/config/default.yaml are unchanged.

Changes

File What changed
lib/templates/docker-compose.project.yml New. Five sidecar services with project network aliases and per-project named volumes
lib/templates/docker-compose.core.yml Removed five sidecar services and their volume definitions
lib/templates/docker-compose.search.yml Removed depends_on (sidecars in separate compose project), added project network
internal/compose/project.go New. ProjectStackUp/Down/Running/DownDirect, ProjectStackEnv
internal/compose/compose.go Added CSPACE_PROJECT_STACK_NAME to ComposeEnv
internal/config/config.go Added ProjectStackName() helper (cspace-<project>-stack)
internal/provision/provision.go New phase 6 "Starting search stack" (17 phases total, shifted from 16)
internal/provision/reporter.go Added "Starting search stack" to Phases slice
internal/cli/down.go Added --project flag; global teardown now discovers and removes project stacks
Tests Updated phase counts (16->17), added ProjectStackEnv/ProjectStackName tests

Lifecycle

  • cspace up <instance> — brings the project stack up first (idempotent), then the instance
  • cspace down <instance> — removes only the instance; project stack stays running
  • cspace down --project — removes all instances + project stack + volumes + network
  • Global teardown removes everything across all projects (including stacks)

Migration

Users on v0.9.5 or earlier with per-instance sidecars:

  • New instances bypass old volumes entirely — they use the project stack
  • Reused running instances keep old per-instance sidecars until torn down (no conflict — project stack is additive)
  • cspace rebuild --reindex forces fresh provisioning, joining the new architecture
  • Old per-instance sidecar volumes orphan naturally and can be cleaned with docker volume prune

Guardrails preserved

Test plan

  • go test ./internal/provision/ — phase count updated to 17, all assertions pass
  • go test ./internal/compose/ — ProjectStackEnv, ComposeEnv with CSPACE_PROJECT_STACK_NAME
  • go test ./internal/config/ — ProjectStackName helper
  • go vet ./... — clean
  • gofmt -l — clean
  • docker compose config renders project template correctly with network aliases and named volumes

elliottregan and others added 7 commits April 22, 2026 04:28
@elliottregan @claude
Add docker-compose.project.yml for project-scoped search sidecars
df43888 
New compose template that declares the five search sidecars (qdrant,
llama-server, llama-clustering, reduce-api, hdbscan-api) at project
scope instead of per-instance. Each service joins the project network
with a plain-name alias so instances resolve them at the same URLs
(http://qdrant:6333, http://llama-server:8080, etc.).

Per-project named volumes (cspace-<project>-qdrant, cspace-<project>-
llama-models) maintain the single-writer invariant from commit 19293ee
while sharing one index across all instances.

Part of #62.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@elliottregan @claude
Move search sidecars from per-instance to project-scoped compose
bb28a24 
Remove the five search sidecar service definitions (qdrant, llama-server,
llama-clustering, reduce-api, hdbscan-api) from docker-compose.core.yml —
they now live in docker-compose.project.yml at project scope.

Also remove the qdrant-storage and llama-models volume declarations from
core.yml (now per-project named volumes in the project stack).

Update docker-compose.search.yml: drop depends_on (qdrant and llama-server
are no longer in the same compose project) and add the project network so
the per-instance search-mcp sidecar can reach the project-scoped sidecars
via their network aliases.

Part of #62.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@elliottregan @claude
Add project stack compose helpers and env plumbing
59c909f 
New internal/compose/project.go with:
  - ProjectStackEnv: env vars for docker-compose.project.yml
  - ProjectStackUp/Down: start/stop the project sidecar stack
  - ProjectStackRunning: check if any project stack container is up
  - ProjectStackDownDirect: teardown by compose name (for --everywhere)

New config.ProjectStackName() returns "cspace-<project>-stack".

Add CSPACE_PROJECT_STACK_NAME to both ProjectStackEnv and the instance
ComposeEnv so the variable is available to all compose templates.

Part of #62.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@elliottregan @claude
Wire project search stack into provisioning as phase 6
307fc70 
Insert "Starting search stack" as phase 6 (between "Creating network"
and "Starting reverse proxy"). All subsequent phases shift by 1, taking
the total from 16 to 17.

The new phase calls compose.ProjectStackUp(cfg), which is idempotent —
if the project stack is already running from a prior instance's boot,
this is a harmless no-op. Failure is a warning, not fatal, since the
search stack is optional infrastructure.

Part of #62.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@elliottregan @claude
Add --project flag to cspace down and stack cleanup to --everywhere
24abda7 
New --project flag tears down all instances for the current project
AND the project-scoped search sidecar stack (plus its volumes and the
project network). This is the full project cleanup command.

Updated --everywhere to also discover and tear down project stacks
(compose projects matching cspace-*-stack) and clean up project
networks for each unique project found.

Part of #62.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@elliottregan @claude
Add tests for project-scoped sidecar changes
c26fa9d 
- Update TestPhasesReference: 16 -> 17 phases, verify new "Starting
  search stack" at index 5
- Add TestProjectStackEnv: verify all env vars for the project stack
- Add TestProjectStackEnvMatchesInstanceEnv: verify shared keys
  (CSPACE_PROJECT_NETWORK, CSPACE_HOME, CSPACE_PROJECT_STACK_NAME) are
  consistent between project and instance envs
- Add ProjectStackName and ProjectNetwork to TestHelpers
- Add CSPACE_PROJECT_STACK_NAME to TestComposeEnv

Part of #62.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@elliottregan @claude
Fix TestBootstrapSearchParam for 17-phase provision
6addcac 
Update the phase count assertion from 16 to 17 and the index from
Phases[15] to Phases[16] to match the new phase structure.

Part of #62.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@netlify
Copy link
Copy Markdown

netlify Bot commented Apr 22, 2026 •
edited
Loading

✅ Deploy Preview for cspace-cli canceled.

Name Link
🔨 Latest commit 6addcac
🔍 Latest deploy log https://app.netlify.com/projects/cspace-cli/deploys/69e850a5a3203f0008e397c2

@elliottregan elliottregan merged commit 549aaf7 into main Apr 22, 2026
9 of 10 checks passed
@elliottregan elliottregan deleted the feat/project-scoped-sidecars branch April 22, 2026 05:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Project-scoped search sidecars + shared index cache across instances

1 participant

@elliottregan