Test Infrastructure

Test Infrastructure

Test Suite

833 tests passing, 3 skipped (Docker-specific)

Layer	Count	Purpose	When to Run
Unit tests	~800	Logic validation for analyzer, parser, pipeline, resolver	Every commit
API contract tests	~30	Response shape, status codes, rate limiter behavior	Every commit
Journey tests	Planned	End-to-end user flows via Playwright	Before merge
Docker tests	3 (skipped)	Deployment-specific (PUID/PGID, socket, healthcheck)	Pre-release

Running Tests

cd C:\Projects\maparr
python -m pytest tests/ -q              # All tests
python -m pytest tests/ -q --tb=short   # With short tracebacks
python -m pytest tests/e2e/ -v          # API contract tests only

Test Stacks

Deterministic compose files covering every conflict scenario. Located in test-stacks/.

Directory Structure

test-stacks/
  single/          # One compose file per scenario
    A01-A06        # Category A: Path conflicts
    B01-B07        # Category B: Permissions
    C01-C05        # Category C: Infrastructure
    D01            # Category D: Observations
    M01-M05        # Multi-category combos + healthy baselines
    F01-F03        # Image family variants
    E01-E05        # Edge cases (env vars, override, malformed, empty, single service)
  clusters/        # Multi-folder layouts (one service per compose file)
    CL01-CL03     # Broken paths, healthy, permissions
  paste/           # Compose files paired with error text for paste flow
    P01-P07       # Import failed, EXDEV, permission denied, RPM, stuck, multi-error
  roots/           # Realistic mixed root directory

Key Test Stacks

Stack	What It Tests
A01	Separate mount trees — the #1 real-world problem
A06	RPM-possible — host paths share common parent (RPM wizard should show)
M03	Mega chaos — 8 services, every category fires
M04	All healthy — zero false positive baseline
E03	Malformed YAML — graceful error handling
F03	Custom unknown images — keyword-based service classification
CL01	Cluster layout — cross-folder path detection
P02	EXDEV paste — service inference without explicit name

Backup Files

Every test stack has a .bak copy of its original compose file. After Apply Fix modifies the file during testing, restore with:

# Restore all stacks from backup
cd test-stacks
for bak in $(find . -name "*.bak" -type f); do
  cp "$bak" "${bak%.bak}"
done

_EXPECT.txt Files

Each stack has a _EXPECT.txt documenting expected behavior:

CONFLICTS: no_shared_mount (high)
HEALTH: red
TABS: Path Fix
APPLY_FIX: yes
RPM_WIZARD: no

Manual Testing Forms

Round 1 (v1.5.1)

• File: tools/testing-form.html
• Scenarios: 42
• Results: 14 pass, 28 fail → 7 bugs found, 6 test expectation errors
• localStorage key: maparr_testing_v2

Round 2 (v1.5.2)

• File: tools/testing-form-v2.html
• Scenarios: 30 across 4 sections:
  • Bug Fix Verification (11 tests)
  • New Feature Validation (8 tests)
  • Regression Spot-Checks (7 tests)
  • Previously Untested (4 tests)
• localStorage key: maparr_testing_v152
• Export: JSON with section breakdown, bug references, per-check granularity

Testing Workflow

1. Start fresh MapArr server: python -m uvicorn backend.main:app --port 9494
2. Restore test stacks from backups
3. Open testing form in browser
4. Work through each scenario, mark pass/fail/skip
5. Export results as JSON
6. Feed JSON to Edmund for analysis and bug fix planning

Lessons Learned (2026-03-10)

• Unit tests validate logic, not integration. 682 unit tests missed bugs that 5 minutes of manual testing caught.
• Layer 2 (Journeys) is the most valuable. Encodes "I click X, I expect Y" — catches real integration bugs.
• Synthetic test stacks are the real asset. Any new analysis logic gets validated against every scenario automatically.
• New features need journey tests FIRST — catches broken assumptions before manual testing.