Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,9 @@ dist-test/
npm-debug.log*
.env
*.tgz

.opencode/
!.opencode/config.json

# Agent guidelines (local development instructions for AI agents)
AGENT.md
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,27 @@ Format follows [Keep a Changelog](https://keepachangelog.com/). Versions follow

---

## [0.2.5] - 2026-03-27

### Added

- Similarity-based duplicate flagging during auto-capture: new captures are checked against existing memories in the same scope using cosine similarity.
- `DedupConfig` with `enabled`, `writeThreshold` (default: 0.92), and `consolidateThreshold` (default: 0.95) for controlling dedup behavior.
- `memory_consolidate` tool: manually triggers merge of similar memories within a scope.
- `memory_consolidate_all` tool: consolidates duplicates across global and project scopes.
- `isPotentialDuplicate` and `duplicateOf` fields in `MemoryRecord.metadata` for tracking potential duplicates.
- `EffectivenessSummary.duplicates` section with `flaggedCount` and `consolidatedCount` for observability.
- `consolidateDuplicates()` store method: merges similar memory pairs where cosine similarity >= consolidateThreshold.
- Pruning preserves newest flagged duplicates when maxEntries forces deletion.

### Changed

- Capture events now include `skipReason: "duplicate-similarity"` when a new memory exceeds writeThreshold.
- `summarizeEvents()` returns counts of flagged and consolidated memories.
- Search results exclude `status=merged` records from display.

---

## [0.2.4] - 2026-03-25

### Added
Expand Down
89 changes: 89 additions & 0 deletions DOCKER_DEV.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Docker Dev Environment for lancedb-opencode-pro Plugin

Run OpenCode in a Docker container with the plugin loaded for isolated testing.

## Architecture

```mermaid
flowchart TB
subgraph Host["Host Machine"]
subgraph Docker["Docker Container (opencode-dev)"]
OpenCode["opencode serve<br/>--port 4096 --hostname 0.0.0.0"]
Plugin["Plugin: file:///workspace/dist/index.js"]
end
HostOpenCode["Host OpenCode (TUI)<br/>~/.config/opencode/opencode.json → attach:4096"]
Ollama["Host Ollama<br/>http://host.docker.internal:11434"]
end

Docker -- "port 4096" --> HostOpenCode
Docker --> Ollama
```

## Setup

### 1. Build and start

```bash
docker compose build --no-cache && docker compose up -d
```

### 2. Build the plugin (first time or after code changes)

```bash
docker compose exec opencode-dev npm run build
```

### 3. Configure Host OpenCode to attach

Create `~/.config/opencode/opencode.json` on your host:

```json
{
"$schema": "https://opencode.ai/config.json",
"agent": {
"attach": "http://localhost:4096"
}
}
```

### 4. Start Host OpenCode

```bash
opencode
```

## Workflow

### Making changes

1. Edit code on host
2. Rebuild: `docker compose exec opencode-dev npm run build`
3. Reload: `docker compose restart opencode-dev`

### Running tests

```bash
docker compose exec opencode-dev npm run verify
docker compose exec opencode-dev npm run test:foundation
```

## Verification

```bash
# Check plugin config is loaded
curl -s -u opencode:devonly http://localhost:4096/config | \
python3 -c "import sys,json; c=json.load(sys.stdin); print('plugin:', c.get('plugin',[]))"

# Check tools are registered
curl -s -u opencode:devonly http://localhost:4096/experimental/tool/ids | \
python3 -c "import sys,json; t=json.load(sys.stdin); print([x for x in t if 'memory' in x])"
```

Expected tool list includes: `memory_search`, `memory_consolidate`, `memory_stats`, etc.

## Cleanup

```bash
docker compose down # stop
docker compose down -v # stop + remove volumes
```
23 changes: 23 additions & 0 deletions Dockerfile.opencode
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
FROM ubuntu:latest
ENV DEBIAN_FRONTEND=noninteractive

WORKDIR /workspace

RUN apt-get update && apt-get install -y --no-install-recommends \
curl \
ca-certificates \
git \
&& rm -rf /var/lib/apt/lists/*

RUN curl -fsSL https://opencode.ai/install | bash

ENV PATH="/root/.opencode/bin:${PATH}"

ARG NODE_VERSION=22
RUN curl -fsSL https://deb.nodesource.com/setup_${NODE_VERSION}.x | bash - \
&& apt-get install -y nodejs \
&& rm -rf /var/lib/apt/lists/*

EXPOSE 4096

CMD ["opencode", "serve", "--port", "4096", "--hostname", "0.0.0.0"]
44 changes: 22 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -675,7 +675,7 @@ Use this flow when publishing a new version to npm.

```bash
docker compose build --no-cache && docker compose up -d
docker compose exec app npm run release:check
docker compose exec opencode-dev npm run release:check
```

3. Confirm npm authentication:
Expand Down Expand Up @@ -731,8 +731,8 @@ ls -l dist dist-test/src 2>/dev/null

```bash
docker compose build --no-cache && docker compose up -d
docker compose exec app npm run typecheck
docker compose exec app npm run build
docker compose exec opencode-dev npm run typecheck
docker compose exec opencode-dev npm run build
```

### Running validation inside Docker
Expand All @@ -741,16 +741,16 @@ docker compose exec app npm run build
docker compose build --no-cache && docker compose up -d

# Quick release check
docker compose exec app npm run verify
docker compose exec opencode-dev npm run verify

# Full release gate (includes benchmark + pack)
docker compose exec app npm run verify:full
docker compose exec opencode-dev npm run verify:full

# Individual workflows
docker compose exec app npm run test:foundation
docker compose exec app npm run test:regression
docker compose exec app npm run test:retrieval
docker compose exec app npm run benchmark:latency
docker compose exec opencode-dev npm run test:foundation
docker compose exec opencode-dev npm run test:regression
docker compose exec opencode-dev npm run test:retrieval
docker compose exec opencode-dev npm run benchmark:latency
```

### Operator verification
Expand All @@ -759,15 +759,15 @@ After running `npm run verify:full`, operators can inspect the following:

```bash
# Confirm the packaged build is installable
docker compose exec app ls -la lancedb-opencode-pro-*.tgz
docker compose exec opencode-dev ls -la lancedb-opencode-pro-*.tgz

# Confirm typecheck and build succeeded
docker compose exec app npm run typecheck
docker compose exec app npm run build
docker compose exec opencode-dev npm run typecheck
docker compose exec opencode-dev npm run build

# Check resolved default storage path
docker compose exec app node -e "import('./dist/index.js').then(() => console.log('plugin loaded'))"
docker compose exec app sh -lc 'ls -la ~/.opencode/memory/lancedb 2>/dev/null || echo "No data yet (expected before first use)"'
docker compose exec opencode-dev node -e "import('./dist/index.js').then(() => console.log('plugin loaded'))"
docker compose exec opencode-dev sh -lc 'ls -la ~/.opencode/memory/lancedb 2>/dev/null || echo "No data yet (expected before first use)"'
```

## Long Memory Verification
Expand All @@ -785,14 +785,14 @@ docker compose build --no-cache && docker compose up -d
The E2E script loads `dist/index.js`, so build artifacts must exist first.

```bash
docker compose exec app npm install
docker compose exec app npm run build
docker compose exec opencode-dev npm install
docker compose exec opencode-dev npm run build
```

### 3. Run the built-in end-to-end memory test

```bash
docker compose exec app npm run test:e2e
docker compose exec opencode-dev npm run test:e2e
```

Expected success output:
Expand All @@ -814,7 +814,7 @@ This verifies all of the following in one run:
The E2E script uses `/tmp/opencode-memory-e2e` as its test database path.

```bash
docker compose exec app ls -la /tmp/opencode-memory-e2e
docker compose exec opencode-dev ls -la /tmp/opencode-memory-e2e
```

If files appear in that directory after the E2E run, memory was written to disk instead of only being kept in process memory.
Expand All @@ -830,7 +830,7 @@ When running through the normal plugin config, the default durable storage path
Check it inside the container with:

```bash
docker compose exec app sh -lc 'ls -la ~/.opencode/memory/lancedb'
docker compose exec opencode-dev sh -lc 'ls -la ~/.opencode/memory/lancedb'
```

### 6. Stronger proof: verify retrieval still works after restart
Expand All @@ -839,8 +839,8 @@ Long memory is only convincing if retrieval still works after the runtime is res

```bash
docker compose restart app
docker compose exec app npm run test:e2e
docker compose exec app ls -la /tmp/opencode-memory-e2e
docker compose exec opencode-dev npm run test:e2e
docker compose exec opencode-dev ls -la /tmp/opencode-memory-e2e
```

If the search step still succeeds after restart and the database files remain present, that is strong evidence that the memory is durable.
Expand All @@ -849,7 +849,7 @@ If the search step still succeeds after restart and the database files remain pr

Treat the feature as verified only when all of these are true:

- `docker compose exec app npm run test:e2e` passes
- `docker compose exec opencode-dev npm run test:e2e` passes
- `/tmp/opencode-memory-e2e` contains LanceDB files after the run
- the memory retrieval step still succeeds after container restart
- the configured OpenCode storage path exists when running real plugin integration
Expand Down
25 changes: 20 additions & 5 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,24 @@
services:
app:
opencode-dev:
build:
context: .
dockerfile: Dockerfile
container_name: lancedb-opencode-pro-app
working_dir: /workspace
dockerfile: Dockerfile.opencode
container_name: opencode-dev
ports:
- "4096:4096"
volumes:
- ./:/workspace
- .:/workspace
- opencode-data:/root/.local/share/opencode
environment:
- OLLAMA_BASE_URL=http://host.docker.internal:11434
- LANCEDB_OPENCODE_PRO_OLLAMA_BASE_URL=http://host.docker.internal:11434
- OPENCODE_SERVER_PASSWORD=devonly
- OPENCODE_CONFIG_CONTENT={"plugin":["file:///workspace/dist/index.js"]}
extra_hosts:
- "host.docker.internal:host-gateway"
stdin_open: true
tty: true
restart: unless-stopped

volumes:
opencode-data:
35 changes: 35 additions & 0 deletions docs/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
| `memory_scope_promote` | Promote memory to global scope | No |
| `memory_scope_demote` | Demote memory from global scope | No |
| `memory_global_list` | List global memories | No |
| `memory_consolidate` | Merge near-duplicate memories in a scope | Yes (`confirm=true`) |
| `memory_consolidate_all` | Consolidate global + current project scope | Yes (`confirm=true`) |

### Common Workflows

Expand All @@ -30,6 +32,39 @@ memory_stats
memory_effectiveness
```

#### Scheduled Consolidation (Daily Cleanup)

The memory system performs opportunistic consolidation on `session.compacted` events. For environments with long-running sessions or infrequent activity, a daily cron job ensures duplicates are merged even without session compaction.

**Example cron script** (`~/.config/opencode/consolidate-cron.sh`):

```bash
#!/bin/bash
# Consolidate duplicates daily at 03:00 UTC
# Run via: opencode --memory-consolidate-all --confirm
#
# Note: Requires OpenCode CLI with lancedb-opencode-pro plugin installed.
# The AI will invoke memory_consolidate_all tool automatically when you ask.

0 3 * * * /path/to/opencode --memory-consolidate-all --confirm >> ~/.opencode/logs/consolidation.log 2>&1
```

**Key configuration options** (via environment or `lancedb-opencode-pro.json`):

| Variable | Default | Description |
|---|---|---|
| `LANCEDB_OPENCODE_PRO_DEDUP_ENABLED` | `true` | Enable/disable dedup |
| `LANCEDB_OPENCODE_PRO_DEDUP_WRITE_THRESHOLD` | `0.92` | Similarity threshold for flagging new memories |
| `LANCEDB_OPENCODE_PRO_DEDUP_CONSOLIDATE_THRESHOLD` | `0.95` | Similarity threshold for merging in consolidation |

**Viewing consolidation metrics**:

```bash
memory_effectiveness
# Look for: duplicates.flaggedCount (memories flagged as potential duplicates)
# Look for: duplicates.consolidatedCount (memories merged via consolidation)
```

#### Search and Verify

```bash
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-03-27
Loading
Loading