You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Two invariants from the planning notes must be CI-enforced from 0.2.0 onward:
Portability — pip install agentanvil (no extras) must import every top-level module and run the quickstart via DirectBackend. If any AgentLoom symbol is importable, CI fails.
Determinism — record a run once, replay 100 times, assert byte-for-byte-identical JSON reports. One of those replays runs on a second runner (different OS or different Python minor) and must still match.
Without CI enforcement, either invariant drifts silently the moment a developer imports AgentLoom at module top-level, or a non-deterministic source leaks in (time.time(), uuid4(), unsorted JSON keys).
Proposal
1. Portability invariant job:
# .github/workflows/ci.ymlportability-invariant:
runs-on: ubuntu-lateststeps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5with:
python-version: "3.12"
- name: Install without extrasrun: | python -m venv /tmp/base-venv /tmp/base-venv/bin/pip install -e .
- name: Verify no AgentLoomrun: | /tmp/base-venv/bin/python -c " import importlib try: importlib.import_module('agentloom') raise SystemExit('FAIL: agentloom is importable under no-extras install') except ModuleNotFoundError: pass "
- name: Import every top-level modulerun: | /tmp/base-venv/bin/python -c " import importlib for mod in ['agentanvil', 'agentanvil.core.contracts', 'agentanvil.backends', 'agentanvil.backends.direct', 'agentanvil.runner.subprocess', 'agentanvil.evaluator.objective', 'agentanvil.evaluator.llm_judge', 'agentanvil.reporter', 'agentanvil.cli.main']: importlib.import_module(mod) "
- name: Run quickstart via DirectBackendrun: | cd examples/quickstart-langchain /tmp/base-venv/bin/pip install -r requirements.txt /tmp/base-venv/bin/agentanvil replay --recording recordings/quickstart.json --contract contract.yaml
Description
Two invariants from the planning notes must be CI-enforced from 0.2.0 onward:
pip install agentanvil(no extras) must import every top-level module and run the quickstart viaDirectBackend. If any AgentLoom symbol is importable, CI fails.Without CI enforcement, either invariant drifts silently the moment a developer imports AgentLoom at module top-level, or a non-deterministic source leaks in (
time.time(),uuid4(), unsorted JSON keys).Proposal
1. Portability invariant job:
2. Determinism invariant job:
The matrix forces at least one run on a "second runner" (different OS + minor Python) per PR.
Scope
.github/workflows/ci.yml— two new jobs.tests/ci/test_portability.py— mirror of the workflow for local dev.tests/ci/test_determinism_smoke.py— short smoke test (≤ 5 replays) runnable without CI.scripts/determinism_stress.sh— full 100-replay script for local investigation.Regression tests
portability-invariantCI job green on every PR.determinism-invariantCI job green on every PR across matrix.test_smoke_record_then_replay_5x_identicalNotes
mainbranch + nightly once stable.