Skip to content

feat: add Digital Twin Universe profile for E2E testing#2

Merged
michaeljabbour merged 5 commits intomichaeljabbour:mainfrom
colombod:main
Apr 30, 2026
Merged

feat: add Digital Twin Universe profile for E2E testing#2
michaeljabbour merged 5 commits intomichaeljabbour:mainfrom
colombod:main

Conversation

@colombod
Copy link
Copy Markdown
Contributor

Summary

Adds a reproducible end-to-end test environment for amplifier-bundle-memory using the Amplifier Digital Twin Universe (DTU) framework.

What's included

  • .amplifier/digital-twin-universe/profiles/memory-bundle-e2e.yaml — Incus container profile provisioning Ubuntu 24.04 with mempalace, Amplifier (via Gitea mirror for production-like install), real Anthropic + OpenAI API keys, and a seeded dual-palace for test isolation
  • tests/fixtures/seed-palace/ — Golden palace seed content (project-context files + drawer content) pre-populated at provision time and frozen as a reset snapshot
  • tests/integration/conftest.py — pytest fixtures: reset_palace (autouse, module-scope) and workspace_dir
  • tests/integration/test_smoke.py — 7 subprocess-based smoke tests verifying provisioning: palace dirs exist, mempalace installed, drawers seeded, search works, amplifier installed, reset-palace restores seed, project-context files present
  • docs/development/dtu.md — Developer guide: prerequisites, launch sequence, three usage modes (pytest / interactive / palace inspection), update loop
  • README.md — Added ## Development section linking to the DTU guide

Installation pattern documented

Uses the behaviour-based install approach so the memory stack layers onto any active bundle without replacing it:

# Recommended — works with any active bundle
amplifier bundle add --app \
  git+https://github.com/michaeljabbour/amplifier-bundle-memory@main#subdirectory=behaviors/mempalace.yaml

# Or add to settings.yaml
includes:
  - bundle: git+https://github.com/michaeljabbour/amplifier-bundle-memory@main#subdirectory=behaviors/mempalace.yaml

How to use the DTU

See docs/development/dtu.md for full instructions. Quick start:

# Mirror the repo to local Gitea (one-time setup)
amplifier-gitea mirror michaeljabbour/amplifier-bundle-memory

# Launch the DTU
amplifier-digital-twin launch .amplifier/digital-twin-universe/profiles/memory-bundle-e2e.yaml \
  --var GITEA_URL=http://localhost:10110 \
  --var GITEA_TOKEN=$(amplifier-gitea token <id> | jq -r .token)

# Run smoke tests inside the container
amplifier-digital-twin exec <id> -- pytest tests/integration/ -v

Test results

  • Existing unit tests: 31 passed ✓
  • Integration smoke tests: DTU-only (require live container) — documented as manual verification step

Diego Colombo added 5 commits April 29, 2026 18:20
Add 6 fixture files under tests/fixtures/seed-palace/ to support the
memory-bundle-e2e DTU profile end-to-end tests.

Files added:
- README.md: Documents the fixture purpose, seeding flow, reset-palace
  script, extension guide, and a files table.
- content/session-notes.md: 600+ word synthetic session notes with
  keywords for capture category detection (decided, resolved, learned,
  pattern). Covers dual-palace pattern, allow_uv_github_fast_path fix,
  project-context discovery, spool requirements, and real API keys.
- content/architecture-decisions.md: 800+ word ADR log covering six
  decisions: two-layer architecture, behaviour-based bundle install,
  hot-path/drain-thread split, dual-palace seeding, real API keys, and
  Gitea mirror.
- project-context/HANDOFF.md: Current work snapshot with working-on,
  next steps, key decisions, open items, and session log entry.
- project-context/PROJECT_CONTEXT.md: Project overview with current
  phase, active milestone, team, architecture pointers table, and
  conventions.
- project-context/GLOSSARY.md: Markdown table with 10 domain terms
  (Palace, Drawer, Wing, Room, Briefing, Spool, Seed palace, Drain
  thread, Capture hook, Briefing hook).

The content/ files are mined into the MemPalace palace via
  mempalace mine ... --mode files
The project-context/ files are copied to /workspace/project-context/
for the briefing hook's _find_project_context_dir() to discover.
Add three files that set up the integration test layer for DTU-based testing:

tests/integration/__init__.py
  Empty package marker so pytest recognises the subdirectory as a Python
  package and collects tests from it.

tests/integration/conftest.py
  DTU-only fixtures:
  - reset_palace (scope=module, autouse=True): calls subprocess.run(['reset-palace'])
    before each test module so every module starts from a clean memory palace.
    Calls pytest.fail() with returncode + stderr if the CLI returns non-zero,
    and raises FileNotFoundError naturally on the host (where reset-palace is
    absent) — confirming the fixture is wired even when the DTU tool is missing.
  - workspace_dir: returns Path('/workspace'), the directory inside the DTU
    that contains project-context/ and amplifier-bundle-memory/.

tests/integration/test_smoke.py
  Minimal probe test that asserts str(workspace_dir) == '/workspace'.
  On the host this fails at reset_palace (FileNotFoundError: 'reset-palace')
  which confirms the conftest is being picked up correctly.
Replaces the single probe test with 7 integration smoke tests that
exercise the full DTU provisioning surface:

1. test_palace_directory_exists: asserts /root/.mempalace and
   /root/.mempalace-seed are present (provision steps create both).

2. test_mempalace_installed: asserts mempalace --version exits 0
   (confirms CLI is on PATH inside the DTU).

3. test_palace_has_seeded_drawers: calls mempalace mcp --call
   mempalace_status and asserts drawer_count > 0 (seed content loaded).

4. test_seed_content_searchable: calls mempalace mcp --call
   mempalace_search with query 'architecture decisions mempalace' and
   asserts at least one result is returned (seed content is queryable).

5. test_amplifier_installed: asserts amplifier --version exits 0
   (confirms Amplifier CLI is installed in the container).

6. test_reset_palace_restores_seed: writes a sentinel file into
   /root/.mempalace, calls reset-palace, verifies the sentinel is gone
   (palace was replaced, not patched), and verifies drawer_count > 0
   after reset (seed restore is functional).

7. test_project_context_files_present: asserts HANDOFF.md,
   PROJECT_CONTEXT.md, and GLOSSARY.md exist under
   /workspace/project-context (provision step 8 populated these files).

All tests are DTU-only: they depend on /root/.mempalace, the mempalace
CLI, amplifier, reset-palace, and /workspace/project-context being
present in the container environment. Running on a host machine causes
all tests to error at the autouse reset_palace fixture (FileNotFoundError
on reset-palace) — this is the correct host behaviour confirming the
tests cannot pass outside the DTU.

Verified:
  python3 -m py_compile tests/integration/test_smoke.py  -> syntax OK
  pytest --collect-only -q  -> 7 tests collected
  pytest -v on host  -> all 7 ERROR at fixture setup (correct)
The tests used 'mempalace mcp --call' syntax which does not exist in
mempalace 3.3.3. The mcp subcommand only starts an MCP stdio server.

Replace the affected tests:
- test_palace_has_seeded_drawers: use 'mempalace status' and regex-parse
  the 'N drawers' count from the human-readable output
- test_seed_content_searchable: use 'mempalace search <query> --results N'
  and check for '[1]' result marker in output
- test_reset_palace_restores_seed: same status change

Remove unused 'json' import, add 're' for drawer count parsing.
…ection

- .amplifier/digital-twin-universe/profiles/memory-bundle-e2e.yaml: Ubuntu 24.04 Incus profile
  with Gitea URL rewrite, dual-palace seeding, and real API key passthrough
- docs/development/dtu.md: prerequisites, launch sequence, usage modes, update loop
- README.md: Development section linking to the DTU guide

Enables reproducible E2E testing via: pytest tests/integration/ -v
@michaeljabbour michaeljabbour merged commit e11dd9a into michaeljabbour:main Apr 30, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants