From 1048168d0c00dbf42a961d059d6b752680660756 Mon Sep 17 00:00:00 2001 From: Jacob Repp Date: Wed, 22 Oct 2025 20:02:09 -0700 Subject: [PATCH 1/3] Add modular testing Taskfile infrastructure User request: "create a new branch to test the taskfile testing infrastructure, if possible we should compose the taskfile from some sub task files - the integration testing should probably live in it's own taskfile in the ./testing directory - I want to have a task that will ensure the docker-compose is up and run through each integration test that runs against this set of services" Created modular testing infrastructure with: - testing/Taskfile.yml with comprehensive test management - Infrastructure lifecycle tasks (infra-up, infra-down, infra-status) - Integration test tasks with auto-managed infrastructure - Acceptance test tasks for all patterns - Backend-specific test tasks - MCP integration test tasks - Comprehensive test suites (test-all, test-all-with-mcp) - Quick smoke test for development Updated root Taskfile.yml to include testing/Taskfile.yml via includes All tasks namespaced as test-infra:* for clarity Benefits: - Separation of concerns (build vs testing infrastructure) - Easier maintenance with focused test Taskfile - Automatic infrastructure management with defer cleanup - Supports both individual and comprehensive test execution - Clear documentation in testing/README.md Co-Authored-By: Claude --- Taskfile.yml | 5 + testing/README.md | 286 +++++++++++++++++++++++++++++++++++++ testing/Taskfile.yml | 327 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 618 insertions(+) create mode 100644 testing/README.md create mode 100644 testing/Taskfile.yml diff --git a/Taskfile.yml b/Taskfile.yml index d717b5ad2..e21e988de 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -3,6 +3,11 @@ version: '3' # Prism Data Access Gateway - Task Build System # Replaces Makefile with automatic change detection and parallel execution +includes: + test-infra: + taskfile: ./testing/Taskfile.yml + dir: . + vars: # Build directories (hygienic out-of-source builds) BUILD_DIR: '{{.ROOT_DIR}}/build' diff --git a/testing/README.md b/testing/README.md new file mode 100644 index 000000000..7254c4c9e --- /dev/null +++ b/testing/README.md @@ -0,0 +1,286 @@ +# Testing Infrastructure - Task Management + +This directory contains the modular Taskfile for managing integration test infrastructure and test execution. + +## Overview + +The `testing/Taskfile.yml` provides a comprehensive suite of tasks for: + +- **Infrastructure Management**: Starting/stopping docker-compose test environments +- **Integration Tests**: Running proxy-pattern lifecycle tests +- **Acceptance Tests**: Running pattern-level acceptance tests +- **Backend Tests**: Running backend-specific integration tests +- **MCP Tests**: Running MCP integration tests + +## Architecture + +The Taskfile is organized into logical sections: + +``` +testing/ +├── Taskfile.yml # Modular task definitions +└── README.md # This file +``` + +All tasks are namespaced under `test-infra:` when included in the root Taskfile: + +```bash +task test-infra:infra-up # Start infrastructure +task test-infra:test-all # Run all tests +task test-infra:infra-down # Stop infrastructure +``` + +## Quick Start + +### Start Test Infrastructure + +```bash +# Start full test infrastructure (Prism + all backends) +task test-infra:infra-up + +# Check status +task test-infra:infra-status + +# View logs +task test-infra:infra-logs +``` + +### Run Tests + +```bash +# Run all integration tests (infrastructure auto-managed) +task test-infra:test-integration-all + +# Run all acceptance tests +task test-infra:test-acceptance-all + +# Run everything (integration + acceptance + backend tests) +task test-infra:test-all +``` + +### Stop Infrastructure + +```bash +# Stop and clean up +task test-infra:infra-down + +# Or clean all test infrastructure +task test-infra:clean-all +``` + +## Available Tasks + +### Infrastructure Management + +| Task | Description | +|------|-------------| +| `infra-up` | Start full test infrastructure (Prism + all backends) | +| `infra-down` | Stop and clean test infrastructure | +| `infra-status` | Check status of running services | +| `infra-logs` | Show logs from all services | +| `infra-logs-proxy` | Show logs from Prism proxy only | +| `infra-logs-admin` | Show logs from Prism admin only | +| `infra-restart` | Restart test infrastructure | + +### MCP Test Infrastructure + +| Task | Description | +|------|-------------| +| `mcp-infra-up` | Start MCP test infrastructure | +| `mcp-infra-down` | Stop MCP test infrastructure | +| `mcp-infra-status` | Check MCP infrastructure status | + +### Integration Tests + +| Task | Description | +|------|-------------| +| `test-integration-lifecycle` | Run lifecycle integration tests | +| `test-integration-dataplane` | Run data plane integration tests | +| `test-integration-shutdown` | Run graceful shutdown tests | +| `test-integration-admin` | Run admin cluster tests (multi-node Raft) | +| `test-integration-all` | Run all integration tests | + +### Acceptance Tests + +| Task | Description | +|------|-------------| +| `test-acceptance-keyvalue` | Run KeyValue pattern tests | +| `test-acceptance-producer` | Run Producer pattern tests | +| `test-acceptance-consumer` | Run Consumer pattern tests | +| `test-acceptance-unified` | Run unified pattern tests | +| `test-acceptance-all` | Run all acceptance tests | + +### Backend Tests + +| Task | Description | +|------|-------------| +| `test-backend-memstore` | Run MemStore backend tests | +| `test-backend-redis` | Run Redis backend tests | +| `test-backends-all` | Run all backend tests | + +### MCP Integration Tests + +| Task | Description | +|------|-------------| +| `test-mcp` | Run MCP integration tests (requires infra) | +| `test-mcp-full` | Run MCP tests with full lifecycle | + +### Comprehensive Suites + +| Task | Description | +|------|-------------| +| `test-all` | Run all integration + acceptance + backend tests | +| `test-all-with-mcp` | Run all tests including MCP | +| `quick-test` | Quick smoke test (short tests only) | + +## Test Infrastructure Services + +The `docker-compose.test.yml` stack includes: + +### Backends +- **PostgreSQL** (port 15432) - KeyValue and relational data +- **Kafka** (port 19092) - TimeSeries and event streaming +- **NATS** (port 4222) - Lightweight messaging +- **Redis** (port 6379) - KeyValue and caching +- **LocalStack** (port 4566) - AWS services (Neptune, S3, KMS) + +### Prism Services +- **Prism Proxy** (port 8980) - High-performance Rust data plane +- **Prism Admin** (port 9000) - Go control plane +- **Prism Canary** - Continuous integration testing client + +## Usage Patterns + +### Development Workflow + +```bash +# 1. Start infrastructure +task test-infra:infra-up + +# 2. Run specific test suite +task test-infra:test-integration-lifecycle + +# 3. Check logs if needed +task test-infra:infra-logs-proxy + +# 4. Stop infrastructure when done +task test-infra:infra-down +``` + +### CI/CD Workflow + +```bash +# Run complete test suite with automatic cleanup +task test-infra:test-all + +# The infra-down cleanup happens automatically via defer +``` + +### Quick Validation + +```bash +# Fast smoke test +task test-infra:quick-test +``` + +## Benefits of Modular Taskfile + +1. **Separation of Concerns**: Testing infrastructure is isolated from build tasks +2. **Easier Maintenance**: Test-related tasks in one file +3. **Reusability**: Can be used standalone or included in root Taskfile +4. **Scalability**: Easy to add new test suites without cluttering root Taskfile +5. **Clarity**: Clear organization of infrastructure vs. test execution + +## Related Documentation + +- [ADR-004: Local-First Testing Strategy](/adr/adr-004) +- [RFC-015: Plugin Acceptance Test Framework](/rfc/rfc-015) +- [Main Taskfile](../Taskfile.yml) +- [Test Directory](../tests/) + +## Direct Usage + +You can also run the testing Taskfile directly: + +```bash +# Run tasks directly from testing directory +task --taskfile testing/Taskfile.yml infra-up +task --taskfile testing/Taskfile.yml test-all +task --taskfile testing/Taskfile.yml infra-down + +# Or show help +task --taskfile testing/Taskfile.yml help +``` + +## Adding New Tests + +To add a new integration test suite: + +1. Create your test in `tests/integration/yourtest/` +2. Add a new task in `testing/Taskfile.yml`: + +```yaml +test-integration-yourtest: + desc: Run your new integration tests + deps: [infra-up] + cmds: + - echo "Running your tests..." + - cd {{.INTEGRATION_TESTS}}/yourtest && go test -v -timeout 5m ./... + - echo "✓ Your tests passed" +``` + +3. Add to the comprehensive suite if needed: + +```yaml +test-integration-all: + deps: [infra-up] + cmds: + - task: test-integration-lifecycle + - task: test-integration-dataplane + - task: test-integration-yourtest # Add here +``` + +## Troubleshooting + +### Infrastructure Won't Start + +```bash +# Check Podman status +task podman-status + +# Ensure Podman machine is running +task podman-start + +# Try restarting infrastructure +task test-infra:infra-restart +``` + +### Tests Timing Out + +```bash +# Check if services are healthy +task test-infra:infra-status + +# View logs to identify issues +task test-infra:infra-logs +``` + +### Cleanup Issues + +```bash +# Force cleanup of all test infrastructure +task test-infra:clean-all + +# If containers are stuck, use podman directly +podman compose -f docker-compose.test.yml down -v --remove-orphans +``` + +## Design Principles + +Following Prism's core design principles: + +1. **Local-First Testing**: Real backends, not mocks (ADR-004) +2. **Automatic Cleanup**: `defer` ensures infrastructure is stopped +3. **Fast Iteration**: Quick smoke tests for rapid development +4. **Comprehensive Coverage**: Full suite for pre-commit validation +5. **Clear Organization**: Modular structure for maintainability diff --git a/testing/Taskfile.yml b/testing/Taskfile.yml new file mode 100644 index 000000000..0160d222f --- /dev/null +++ b/testing/Taskfile.yml @@ -0,0 +1,327 @@ +version: '3' + +# Integration Testing Infrastructure for Prism +# This Taskfile manages docker-compose lifecycle and integration test execution +# +# Usage: +# task --taskfile testing/Taskfile.yml infra-up +# task --taskfile testing/Taskfile.yml test-all +# task --taskfile testing/Taskfile.yml infra-down + +vars: + # Docker compose setup + COMPOSE: podman compose + COMPOSE_FILE: docker-compose.test.yml + COMPOSE_FILE_MCP: tests/testing/mcp/docker-compose.mcp-test.yml + + # Test directories + INTEGRATION_TESTS: tests/integration + ACCEPTANCE_TESTS: tests/acceptance + MCP_TESTS: tests/testing/mcp + + # Build and log directories + BUILD_DIR: '{{.ROOT_DIR}}/build' + TEST_LOGS_DIR: '{{.BUILD_DIR}}/test-logs' + +env: + DOCKER_HOST: '{{.DOCKER_HOST}}' + +tasks: + # ============================================================================ + # Infrastructure Management + # ============================================================================ + + infra-up: + desc: Start integration test infrastructure (Prism + all backends) + cmds: + - echo "Starting integration test infrastructure..." + - '{{.COMPOSE}} -f {{.COMPOSE_FILE}} up -d' + - echo "Waiting for services to be healthy..." + - sleep 15 + - echo "Checking service health..." + - '{{.COMPOSE}} -f {{.COMPOSE_FILE}} ps' + - echo "✓ Integration test infrastructure is ready" + - echo "Prism Proxy - localhost:8980" + - echo "Prism Admin - localhost:9000" + - echo "PostgreSQL - localhost:15432" + - echo "Kafka - localhost:19092" + - echo "NATS - localhost:4222" + - echo "Redis - localhost:6379" + + infra-down: + desc: Stop integration test infrastructure + cmds: + - echo "Stopping integration test infrastructure..." + - '{{.COMPOSE}} -f {{.COMPOSE_FILE}} down -v' + - echo "✓ Infrastructure stopped and cleaned" + + infra-status: + desc: Check status of integration test infrastructure + cmds: + - echo "=== Infrastructure Status ===" + - '{{.COMPOSE}} -f {{.COMPOSE_FILE}} ps' + - echo "" + - echo "=== Service Health ===" + - '{{.COMPOSE}} -f {{.COMPOSE_FILE}} ps --format json | jq -r ".[] | \"\(.Name): \(.State) - \(.Health)\""' + + infra-logs: + desc: Show logs from integration test infrastructure + cmds: + - '{{.COMPOSE}} -f {{.COMPOSE_FILE}} logs -f' + + infra-logs-proxy: + desc: Show logs from Prism proxy only + cmds: + - '{{.COMPOSE}} -f {{.COMPOSE_FILE}} logs -f prism-proxy' + + infra-logs-admin: + desc: Show logs from Prism admin only + cmds: + - '{{.COMPOSE}} -f {{.COMPOSE_FILE}} logs -f prism-admin' + + infra-restart: + desc: Restart integration test infrastructure + deps: [infra-down, infra-up] + + # ============================================================================ + # MCP Test Infrastructure + # ============================================================================ + + mcp-infra-up: + desc: Start MCP test infrastructure (minimal for MCP tests) + cmds: + - echo "Starting MCP test infrastructure..." + - '{{.COMPOSE}} -f {{.COMPOSE_FILE_MCP}} up -d' + - echo "Waiting for services to be healthy..." + - sleep 10 + - '{{.COMPOSE}} -f {{.COMPOSE_FILE_MCP}} ps' + - echo "✓ MCP test infrastructure is ready" + + mcp-infra-down: + desc: Stop MCP test infrastructure + cmds: + - echo "Stopping MCP test infrastructure..." + - '{{.COMPOSE}} -f {{.COMPOSE_FILE_MCP}} down -v' + - echo "✓ MCP infrastructure stopped" + + mcp-infra-status: + desc: Check status of MCP test infrastructure + cmds: + - '{{.COMPOSE}} -f {{.COMPOSE_FILE_MCP}} ps' + + # ============================================================================ + # Integration Tests + # ============================================================================ + + test-integration-lifecycle: + desc: Run lifecycle integration tests (proxy-pattern lifecycle) + deps: [infra-up] + cmds: + - echo "Running lifecycle integration tests..." + - cd {{.INTEGRATION_TESTS}} && go test -v -timeout 5m -run TestLifecycle ./... + - echo "✓ Lifecycle tests passed" + + test-integration-dataplane: + desc: Run data plane integration tests + deps: [infra-up] + cmds: + - echo "Running data plane integration tests..." + - cd {{.INTEGRATION_TESTS}} && go test -v -timeout 5m -run TestDataPlane ./... + - echo "✓ Data plane tests passed" + + test-integration-shutdown: + desc: Run graceful shutdown integration tests + deps: [infra-up] + cmds: + - echo "Running graceful shutdown tests..." + - cd {{.INTEGRATION_TESTS}}/shutdown && go test -v -timeout 5m ./... + - echo "✓ Graceful shutdown tests passed" + + test-integration-admin: + desc: Run admin cluster integration tests (multi-node Raft) + cmds: + - echo "Running admin cluster integration tests..." + - echo "Note - This test manages its own docker-compose stack" + - cd {{.INTEGRATION_TESTS}}/admin && go test -v -timeout 15m ./... + - echo "✓ Admin cluster tests passed" + + test-integration-all: + desc: Run all integration tests with infrastructure + deps: [infra-up] + cmds: + - echo "Running all integration tests..." + - cd {{.INTEGRATION_TESTS}} && go test -v -timeout 10m ./... + - defer: { task: infra-down } + - echo "✓ All integration tests passed" + + # ============================================================================ + # MCP Integration Tests + # ============================================================================ + + test-mcp: + desc: Run MCP integration tests + deps: [mcp-infra-up] + dir: '{{.MCP_TESTS}}' + cmds: + - echo "Installing MCP test dependencies..." + - npm install --silent + - echo "Running MCP integration tests..." + - npm test + - echo "✓ MCP integration tests passed" + + test-mcp-full: + desc: Run MCP tests with full lifecycle (start infra, test, stop) + cmds: + - task: mcp-infra-up + - task: test-mcp + - defer: { task: mcp-infra-down } + + # ============================================================================ + # Backend-Specific Integration Tests + # ============================================================================ + + test-backend-redis: + desc: Run Redis backend integration tests + deps: [infra-up] + cmds: + - echo "Running Redis backend tests..." + - cd tests/backends/redis && go test -v -timeout 5m ./... + - echo "✓ Redis backend tests passed" + + test-backend-memstore: + desc: Run MemStore backend integration tests + cmds: + - echo "Running MemStore backend tests..." + - cd tests/backends/memstore && go test -v -timeout 5m ./... + - echo "✓ MemStore backend tests passed" + + test-backends-all: + desc: Run all backend-specific integration tests + deps: [infra-up] + cmds: + - echo "Running all backend tests..." + - task: test-backend-memstore + - task: test-backend-redis + - defer: { task: infra-down } + + # ============================================================================ + # Acceptance Tests (Pattern-level) + # ============================================================================ + + test-acceptance-keyvalue: + desc: Run KeyValue pattern acceptance tests + deps: [infra-up] + cmds: + - echo "Running KeyValue acceptance tests..." + - cd {{.ACCEPTANCE_TESTS}}/patterns/keyvalue && go test -v -timeout 10m ./... + - echo "✓ KeyValue acceptance tests passed" + + test-acceptance-producer: + desc: Run Producer pattern acceptance tests + deps: [infra-up] + cmds: + - echo "Running Producer acceptance tests..." + - cd {{.ACCEPTANCE_TESTS}}/patterns/producer && go test -v -timeout 10m ./... + - echo "✓ Producer acceptance tests passed" + + test-acceptance-consumer: + desc: Run Consumer pattern acceptance tests + deps: [infra-up] + cmds: + - echo "Running Consumer acceptance tests..." + - cd {{.ACCEPTANCE_TESTS}}/patterns/consumer && go test -v -timeout 10m ./... + - echo "✓ Consumer acceptance tests passed" + + test-acceptance-unified: + desc: Run unified producer/consumer acceptance tests + deps: [infra-up] + cmds: + - echo "Running unified acceptance tests..." + - cd {{.ACCEPTANCE_TESTS}}/patterns/unified && go test -v -timeout 10m ./... + - echo "✓ Unified acceptance tests passed" + + test-acceptance-all: + desc: Run all acceptance tests with infrastructure + deps: [infra-up] + cmds: + - echo "Running all acceptance tests..." + - task: test-acceptance-keyvalue + - task: test-acceptance-producer + - task: test-acceptance-consumer + - task: test-acceptance-unified + - defer: { task: infra-down } + + # ============================================================================ + # Comprehensive Test Suites + # ============================================================================ + + test-all: + desc: Run all integration and acceptance tests with infrastructure + deps: [infra-up] + cmds: + - echo "Running comprehensive test suite..." + - mkdir -p {{.TEST_LOGS_DIR}} + - task: test-integration-all + - task: test-acceptance-all + - task: test-backends-all + - defer: { task: infra-down } + - echo "✓ All tests passed!" + + test-all-with-mcp: + desc: Run all tests including MCP integration tests + cmds: + - echo "Running complete test suite with MCP..." + - task: test-all + - task: test-mcp-full + - echo "✓ Complete test suite passed!" + + # ============================================================================ + # Quick Test Tasks (for development) + # ============================================================================ + + quick-test: + desc: Quick smoke test with infrastructure (basic integration only) + deps: [infra-up] + cmds: + - echo "Running quick smoke test..." + - cd {{.INTEGRATION_TESTS}} && go test -v -timeout 2m -short ./... + - defer: { task: infra-down } + - echo "✓ Quick test passed" + + # ============================================================================ + # Cleanup and Utilities + # ============================================================================ + + clean-all: + desc: Stop all test infrastructure and clean up + cmds: + - echo "Cleaning up all test infrastructure..." + - task: infra-down + - task: mcp-infra-down + - echo "✓ All test infrastructure cleaned" + + help: + desc: Show available testing tasks + cmds: + - echo "Integration Testing Infrastructure" + - echo "" + - echo "Infrastructure Management" + - echo " infra-up - Start full test infrastructure" + - echo " infra-down - Stop full test infrastructure" + - echo " infra-status - Check infrastructure status" + - echo " infra-logs - View all infrastructure logs" + - echo " mcp-infra-up - Start MCP test infrastructure" + - echo " mcp-infra-down - Stop MCP test infrastructure" + - echo "" + - echo "Integration Tests" + - echo " test-integration-all - Run all integration tests" + - echo " test-acceptance-all - Run all acceptance tests" + - echo " test-backends-all - Run all backend tests" + - echo " test-mcp-full - Run MCP integration tests" + - echo "" + - echo "Comprehensive Suites" + - echo " test-all - Run all tests with infrastructure" + - echo " test-all-with-mcp - Run all tests including MCP" + - echo " quick-test - Quick smoke test" + - echo "" + - echo "For full list - task --taskfile testing/Taskfile.yml --list" From ce2aba58b5c36a8db33d0b9b86f300c09d68b2ea Mon Sep 17 00:00:00 2001 From: Jacob Repp Date: Wed, 22 Oct 2025 20:27:59 -0700 Subject: [PATCH 2/3] Simplify testing namespace from test-infra to test User request: "let's simplify the tasks so it's just test: for the namespace" Changed namespace from test-infra to test for cleaner, more intuitive usage: - Updated root Taskfile.yml includes section - Updated all documentation in testing/README.md - All testing tasks now accessible as task test:* Examples: - task test:infra-up (was test-infra:infra-up) - task test:test-all (was test-infra:test-all) - task test:help (was test-infra:help) Benefits: - Shorter, cleaner namespace - More intuitive for users - Consistent with common task naming patterns Co-Authored-By: Claude --- Taskfile.yml | 2 +- testing/README.md | 44 ++++++++++++++++++++++---------------------- 2 files changed, 23 insertions(+), 23 deletions(-) diff --git a/Taskfile.yml b/Taskfile.yml index e21e988de..139dc035f 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -4,7 +4,7 @@ version: '3' # Replaces Makefile with automatic change detection and parallel execution includes: - test-infra: + test: taskfile: ./testing/Taskfile.yml dir: . diff --git a/testing/README.md b/testing/README.md index 7254c4c9e..c426690a4 100644 --- a/testing/README.md +++ b/testing/README.md @@ -22,12 +22,12 @@ testing/ └── README.md # This file ``` -All tasks are namespaced under `test-infra:` when included in the root Taskfile: +All tasks are namespaced under `test:` when included in the root Taskfile: ```bash -task test-infra:infra-up # Start infrastructure -task test-infra:test-all # Run all tests -task test-infra:infra-down # Stop infrastructure +task test:infra-up # Start infrastructure +task test:test-all # Run all tests +task test:infra-down # Stop infrastructure ``` ## Quick Start @@ -36,36 +36,36 @@ task test-infra:infra-down # Stop infrastructure ```bash # Start full test infrastructure (Prism + all backends) -task test-infra:infra-up +task test:infra-up # Check status -task test-infra:infra-status +task test:infra-status # View logs -task test-infra:infra-logs +task test:infra-logs ``` ### Run Tests ```bash # Run all integration tests (infrastructure auto-managed) -task test-infra:test-integration-all +task test:test-integration-all # Run all acceptance tests -task test-infra:test-acceptance-all +task test:test-acceptance-all # Run everything (integration + acceptance + backend tests) -task test-infra:test-all +task test:test-all ``` ### Stop Infrastructure ```bash # Stop and clean up -task test-infra:infra-down +task test:infra-down # Or clean all test infrastructure -task test-infra:clean-all +task test:clean-all ``` ## Available Tasks @@ -155,23 +155,23 @@ The `docker-compose.test.yml` stack includes: ```bash # 1. Start infrastructure -task test-infra:infra-up +task test:infra-up # 2. Run specific test suite -task test-infra:test-integration-lifecycle +task test:test-integration-lifecycle # 3. Check logs if needed -task test-infra:infra-logs-proxy +task test:infra-logs-proxy # 4. Stop infrastructure when done -task test-infra:infra-down +task test:infra-down ``` ### CI/CD Workflow ```bash # Run complete test suite with automatic cleanup -task test-infra:test-all +task test:test-all # The infra-down cleanup happens automatically via defer ``` @@ -180,7 +180,7 @@ task test-infra:test-all ```bash # Fast smoke test -task test-infra:quick-test +task test:quick-test ``` ## Benefits of Modular Taskfile @@ -252,24 +252,24 @@ task podman-status task podman-start # Try restarting infrastructure -task test-infra:infra-restart +task test:infra-restart ``` ### Tests Timing Out ```bash # Check if services are healthy -task test-infra:infra-status +task test:infra-status # View logs to identify issues -task test-infra:infra-logs +task test:infra-logs ``` ### Cleanup Issues ```bash # Force cleanup of all test infrastructure -task test-infra:clean-all +task test:clean-all # If containers are stuck, use podman directly podman compose -f docker-compose.test.yml down -v --remove-orphans From 08aaffd8171cd83aa3fa0bd7b55ff105544f1501 Mon Sep 17 00:00:00 2001 From: Jacob Repp Date: Wed, 22 Oct 2025 20:37:16 -0700 Subject: [PATCH 3/3] Consolidate ALL tests into test namespace User request: "do unit tests fit into this new task namespace? yes" Moved ALL test tasks from root Taskfile.yml to testing/Taskfile.yml under unified test: namespace with clear hierarchy: Unit Tests (Fast, No Dependencies): - unit-all, unit-proxy, unit-core, unit-prismctl, unit-operator - unit-drivers (memstore, redis, nats, kafka, postgres) Integration Tests (Requires Infrastructure): - test-integration-all, integration-rust, integration-k8s - test-integration-lifecycle, test-integration-dataplane, etc. Acceptance Tests: - test-acceptance-all (docker-compose based) - acceptance-testcontainers (testcontainers based) - acceptance-parallel Parallel Execution: - parallel, parallel-fast, parallel-fail-fast Comprehensive Suites: - all (unit + integration + acceptance) - all-with-mcp Changes: - Added all unit tests to testing/Taskfile.yml - Added testcontainers-based acceptance tests - Added Rust and K8s integration tests - Added parallel execution tasks - Removed all 250+ lines of test tasks from root Taskfile.yml - Updated comprehensive suites to include unit tests - Updated testing/README.md with new structure and hierarchy - Updated help task with complete test organization Total: 55 test tasks now consolidated under test: namespace Benefits: - Single namespace for ALL tests (unit, integration, acceptance) - Clear hierarchy: Unit -> Integration -> Acceptance -> All - Fast iteration: Unit tests require no infrastructure - Easy discovery: task --list | grep test: - Consistent naming: Predictable patterns throughout - Root Taskfile focused on builds, not tests Co-Authored-By: Claude --- Taskfile.yml | 293 +------------------------------------------ testing/README.md | 166 ++++++++++++++++++------ testing/Taskfile.yml | 290 ++++++++++++++++++++++++++++++++++++++---- 3 files changed, 396 insertions(+), 353 deletions(-) diff --git a/Taskfile.yml b/Taskfile.yml index 139dc035f..320e53a39 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -317,299 +317,10 @@ tasks: done - echo "✓ Debug builds complete" - # ============================================================================ - # Testing Tasks - # ============================================================================ - - test: - desc: Run all tests (unit, acceptance, integration) - deps: [test-proxy, test-patterns, test-operator, test-acceptance, test-integration-go] - cmds: - - echo "✓ All tests passed" - - test-parallel: - desc: Run all tests in parallel (fast!) - cmds: - - echo "Running tests in parallel..." - - uv run tooling/parallel_test.py --log-dir {{.TEST_LOGS_DIR}} - - echo "✓ Parallel tests complete" - - test-parallel-fast: - desc: Run fast tests only in parallel (skip acceptance) - cmds: - - echo "Running fast tests in parallel..." - - uv run tooling/parallel_test.py --fast --log-dir {{.TEST_LOGS_DIR}} - - echo "✓ Fast parallel tests complete" - - test-parallel-fail-fast: - desc: Run tests in parallel with fail-fast - cmds: - - echo "Running tests in parallel with fail-fast..." - - uv run tooling/parallel_test.py --fail-fast --log-dir {{.TEST_LOGS_DIR}} - - echo "✓ Parallel tests complete" - - test-proxy: - desc: Run Rust proxy unit tests - sources: - - prism-proxy/src/**/*.rs - - prism-proxy/Cargo.toml - cmds: - - echo "Running Rust proxy tests..." - - cd prism-proxy && cargo test --lib - - echo "✓ Proxy unit tests passed" - - test-patterns: - desc: Run all Go pattern tests - deps: [test-core, test-prismctl, test-memstore, test-redis, test-nats, test-kafka, test-postgres] - - test-core: - desc: Run Core SDK tests - sources: - - pkg/plugin/**/*.go - cmds: - - echo "Running Core SDK tests..." - - cd pkg/plugin && go test -v -cover ./... - - echo "✓ Core SDK tests passed" - - test-prismctl: - desc: Run prismctl tests - sources: - - cmd/prismctl/**/*.go - cmds: - - echo "Running prismctl tests..." - - cd cmd/prismctl && go test -v -cover ./... - - echo "✓ prismctl tests passed" - - test-memstore: - desc: Run MemStore driver tests - sources: - - pkg/drivers/memstore/**/*.go - cmds: - - echo "Running memstore tests..." - - cd pkg/drivers/memstore && go test -v -cover ./... - - echo "✓ memstore tests passed" - - test-redis: - desc: Run Redis driver tests - sources: - - pkg/drivers/redis/**/*.go - cmds: - - echo "Running redis tests..." - - cd pkg/drivers/redis && go test -v -cover ./... - - echo "✓ redis tests passed" - - test-nats: - desc: Run NATS driver tests - sources: - - pkg/drivers/nats/**/*.go - cmds: - - echo "Running nats tests..." - - cd pkg/drivers/nats && go test -v -cover ./... - - echo "✓ nats tests passed" - - test-kafka: - desc: Run Kafka driver tests - sources: - - pkg/drivers/kafka/**/*.go - cmds: - - echo "Running kafka tests..." - - cd pkg/drivers/kafka && go test -v -cover ./... - - echo "✓ kafka tests passed" - - test-postgres: - desc: Run Postgres driver tests - sources: - - pkg/drivers/postgres/**/*.go - cmds: - - echo "Running postgres tests..." - - cd pkg/drivers/postgres && go test -v -cover ./... - - echo "✓ postgres tests passed" - - test-operator: - desc: Run Kubernetes operator controller tests - sources: - - prism-operator/controllers/**/*.go - - prism-operator/api/**/*.go - cmds: - - echo "Running operator controller tests..." - - cd prism-operator/controllers && go test -v -cover ./... - - echo "✓ Operator controller tests passed" - - test-integration: - desc: Run integration tests (requires built binaries) - sources: - - prism-proxy/tests/**/*.rs - cmds: - - echo "Running integration tests..." - - cd prism-proxy && cargo test --test integration_test -- --ignored --nocapture - - echo "✓ Integration tests passed" - - test-all: - desc: Run all tests (unit, integration, acceptance) - deps: [test, test-integration, test-integration-go, test-acceptance] - cmds: - - echo "✓ All tests (unit + integration + acceptance) passed" - - test-acceptance: - desc: Run all acceptance tests with testcontainers - deps: - - test-acceptance-interfaces - - test-acceptance-redis - - test-acceptance-nats - - test-acceptance-kafka - - test-acceptance-postgres - cmds: - - echo "✓ All acceptance tests passed" - - test-acceptance-interfaces: - desc: Run interface-based acceptance tests - sources: - - tests/acceptance/interfaces/**/*.go - cmds: - - echo "Running interface-based acceptance tests..." - - cd tests/acceptance/interfaces && go test -v -timeout 10m ./... - - echo "✓ Interface-based acceptance tests passed" - - test-acceptance-redis: - desc: Run Redis acceptance tests - sources: - - tests/acceptance/redis/**/*.go - cmds: - - echo "Running redis acceptance tests..." - - cd tests/acceptance/redis && go test -v -timeout 10m ./... - - echo "✓ redis acceptance tests passed" - - test-acceptance-nats: - desc: Run NATS acceptance tests - sources: - - tests/acceptance/nats/**/*.go - cmds: - - echo "Running nats acceptance tests..." - - cd tests/acceptance/nats && go test -v -timeout 10m ./... - - echo "✓ nats acceptance tests passed" - - test-acceptance-kafka: - desc: Run Kafka acceptance tests - sources: - - tests/acceptance/kafka/**/*.go - cmds: - - echo "Running kafka acceptance tests..." - - cd tests/acceptance/kafka && go test -v -timeout 10m ./... - - echo "✓ kafka acceptance tests passed" - - test-acceptance-postgres: - desc: Run Postgres acceptance tests - sources: - - tests/acceptance/postgres/**/*.go - cmds: - - echo "Running postgres acceptance tests..." - - cd tests/acceptance/postgres && go test -v -timeout 10m ./... - - echo "✓ postgres acceptance tests passed" - - test-acceptance-quiet: - desc: Run all acceptance tests in quiet mode - cmds: - - echo "Running acceptance tests in quiet mode..." - - PRISM_TEST_QUIET=1 task test-acceptance - - echo "✓ All acceptance tests passed (quiet mode)" - - test-acceptance-parallel: - desc: Run acceptance tests in parallel with matrix report - cmds: - - echo "Running acceptance tests in parallel..." - - mkdir -p {{.BUILD_DIR}}/reports - - uv run tooling/parallel_acceptance_test.py - - echo "✓ Parallel acceptance tests complete" - - test-integration-go: - desc: Run Go integration tests (proxy-pattern lifecycle) - sources: - - tests/integration/**/*.go - cmds: - - echo "Running Go integration tests..." - - cd tests/integration && go test -v -timeout 5m ./... - - echo "✓ Go integration tests passed" - - test-integration-admin: - desc: Run admin cluster integration tests (multi-node Raft) - deps: [prism-admin] - sources: - - tests/integration/admin/**/*.go - cmds: - - echo "Running admin cluster integration tests..." - - cd tests/integration/admin && go test -v -timeout 15m ./... - - echo "✓ Admin cluster integration tests passed" - - test-integration-shutdown: - desc: Run graceful shutdown integration tests (RFC-041) - sources: - - tests/integration/shutdown/**/*.go - cmds: - - echo "Running graceful shutdown integration tests..." - - cd tests/integration/shutdown && go test -v -timeout 5m ./... - - echo "✓ Graceful shutdown integration tests passed" - - test-integration-k8s: - desc: Run Kubernetes integration tests (requires local K8s cluster and built images) - sources: - - tests/integration/k8s/**/*.go - - prism-operator/api/**/*.go - - prism-operator/controllers/**/*.go - cmds: - - echo "Running Kubernetes integration tests..." - - echo "Prerequisites:" - - echo " - K8s cluster running (kubectl cluster-info)" - - echo " - Docker images built (task k8s-build-images)" - - cd tests/integration/k8s && go test -v -timeout 30m ./... - - echo "✓ Kubernetes integration tests passed" - - test-integration-k8s-short: - desc: Run quick Kubernetes integration tests (minimal test only) - cmds: - - echo "Running quick Kubernetes integration tests..." - - cd tests/integration/k8s && go test -v -timeout 10m -short ./... - - echo "✓ Quick Kubernetes integration tests passed" - - # ============================================================================ - # MCP Integration Tests - # ============================================================================ - - test-infra-up: - desc: Start docker-compose test infrastructure (Prism proxy + backends) - cmds: - - echo "Starting test infrastructure..." - - '{{.COMPOSE}} -f docker-compose.test.yml up -d' - - echo "Waiting for services to be healthy..." - - sleep 10 - - echo "✓ Test infrastructure running" - - test-infra-down: - desc: Stop docker-compose test infrastructure - cmds: - - echo "Stopping test infrastructure..." - - '{{.COMPOSE}} -f docker-compose.test.yml down -v' - - echo "✓ Test infrastructure stopped" - - test-mcp: - desc: Run MCP integration tests (requires test infrastructure) - deps: [test-infra-up] - dir: tests/testing/mcp - cmds: - - echo "Installing MCP test dependencies..." - - npm install --silent - - echo "Running MCP integration tests..." - - npm test - - echo "✓ MCP integration tests passed" - - test-mcp-full: - desc: Run MCP tests with full lifecycle (start infra, test, stop) - cmds: - - task test-infra-up - - task test-mcp - - defer: { task: test-infra-down } - # ============================================================================ # Code Coverage Tasks + # NOTE: All test tasks have moved to testing/Taskfile.yml (test: namespace) + # Use: task test:unit-all, task test:integration-all, task test:all # ============================================================================ coverage: diff --git a/testing/README.md b/testing/README.md index c426690a4..9f68b0003 100644 --- a/testing/README.md +++ b/testing/README.md @@ -1,16 +1,17 @@ -# Testing Infrastructure - Task Management +# Testing Infrastructure - Unified Test Namespace -This directory contains the modular Taskfile for managing integration test infrastructure and test execution. +This directory contains the consolidated Taskfile for ALL Prism tests under the `test:` namespace. ## Overview -The `testing/Taskfile.yml` provides a comprehensive suite of tasks for: +The `testing/Taskfile.yml` consolidates ALL test tasks into a single, organized namespace: +- **Unit Tests**: Fast tests with no external dependencies (drivers, SDK, proxy) +- **Integration Tests**: Tests requiring docker-compose infrastructure (lifecycle, dataplane) +- **Acceptance Tests**: Pattern-level validation with testcontainers or docker-compose +- **Parallel Execution**: Fast parallel test runs - **Infrastructure Management**: Starting/stopping docker-compose test environments -- **Integration Tests**: Running proxy-pattern lifecycle tests -- **Acceptance Tests**: Running pattern-level acceptance tests -- **Backend Tests**: Running backend-specific integration tests -- **MCP Tests**: Running MCP integration tests +- **MCP Tests**: MCP integration tests ## Architecture @@ -32,30 +33,43 @@ task test:infra-down # Stop infrastructure ## Quick Start -### Start Test Infrastructure +### Run Unit Tests (Fastest) ```bash -# Start full test infrastructure (Prism + all backends) -task test:infra-up - -# Check status -task test:infra-status +# Run all unit tests (no infrastructure needed) +task test:unit-all -# View logs -task test:infra-logs +# Run specific unit tests +task test:unit-proxy # Rust proxy +task test:unit-core # Core SDK +task test:unit-drivers # All drivers ``` -### Run Tests +### Run Integration/Acceptance Tests ```bash -# Run all integration tests (infrastructure auto-managed) +# Infrastructure is auto-managed (starts/stops automatically) task test:test-integration-all - -# Run all acceptance tests task test:test-acceptance-all -# Run everything (integration + acceptance + backend tests) -task test:test-all +# Or run everything at once +task test:all +``` + +### Manual Infrastructure Management + +```bash +# Start infrastructure manually +task test:infra-up + +# Check status +task test:infra-status + +# View logs +task test:infra-logs + +# Stop infrastructure +task test:infra-down ``` ### Stop Infrastructure @@ -68,8 +82,58 @@ task test:infra-down task test:clean-all ``` +## Test Hierarchy + +``` +test: +├── Unit Tests (Fast, No Dependencies) +│ ├── unit-all +│ ├── unit-proxy (Rust proxy unit tests) +│ ├── unit-core (Core SDK tests) +│ ├── unit-prismctl +│ ├── unit-operator (K8s operator) +│ └── unit-drivers (memstore, redis, nats, kafka, postgres) +│ +├── Integration Tests (Requires Infrastructure) +│ ├── test-integration-all +│ ├── integration-rust +│ ├── integration-k8s +│ └── integration-k8s-short +│ +├── Acceptance Tests +│ ├── test-acceptance-all (docker-compose based) +│ ├── acceptance-testcontainers (testcontainers based) +│ └── acceptance-parallel +│ +├── Parallel Execution +│ ├── parallel (all tests) +│ ├── parallel-fast (skip acceptance) +│ └── parallel-fail-fast +│ +└── Comprehensive Suites + ├── all (unit + integration + acceptance) + ├── all-with-mcp + └── quick-test (smoke test) +``` + ## Available Tasks +### Unit Tests + +| Task | Description | +|------|-------------| +| `unit-all` | Run all unit tests | +| `unit-proxy` | Run Rust proxy unit tests | +| `unit-core` | Run Core SDK unit tests | +| `unit-prismctl` | Run prismctl unit tests | +| `unit-operator` | Run Kubernetes operator unit tests | +| `unit-drivers` | Run all driver unit tests | +| `unit-memstore` | Run MemStore driver tests | +| `unit-redis` | Run Redis driver tests | +| `unit-nats` | Run NATS driver tests | +| `unit-kafka` | Run Kafka driver tests | +| `unit-postgres` | Run Postgres driver tests | + ### Infrastructure Management | Task | Description | @@ -125,12 +189,21 @@ task test:clean-all | `test-mcp` | Run MCP integration tests (requires infra) | | `test-mcp-full` | Run MCP tests with full lifecycle | +### Parallel Execution + +| Task | Description | +|------|-------------| +| `parallel` | Run all tests in parallel | +| `parallel-fast` | Run fast tests only in parallel (skip acceptance) | +| `parallel-fail-fast` | Run tests in parallel with fail-fast | +| `acceptance-parallel` | Run acceptance tests in parallel with matrix report | + ### Comprehensive Suites | Task | Description | |------|-------------| -| `test-all` | Run all integration + acceptance + backend tests | -| `test-all-with-mcp` | Run all tests including MCP | +| `all` | Run ALL tests (unit + integration + acceptance) | +| `all-with-mcp` | Run all tests including MCP | | `quick-test` | Quick smoke test (short tests only) | ## Test Infrastructure Services @@ -151,29 +224,38 @@ The `docker-compose.test.yml` stack includes: ## Usage Patterns -### Development Workflow +### Fast Development Workflow (Unit Tests) ```bash -# 1. Start infrastructure -task test:infra-up +# Run unit tests only (instant, no infrastructure) +task test:unit-all -# 2. Run specific test suite -task test:test-integration-lifecycle +# Or specific unit tests +task test:unit-core +task test:unit-redis +``` -# 3. Check logs if needed -task test:infra-logs-proxy +### Full Development Workflow -# 4. Stop infrastructure when done -task test:infra-down +```bash +# 1. Run unit tests first (fast feedback) +task test:unit-all + +# 2. Run integration tests (infrastructure auto-managed) +task test:test-integration-all + +# 3. Run acceptance tests +task test:test-acceptance-all ``` ### CI/CD Workflow ```bash # Run complete test suite with automatic cleanup -task test:test-all +task test:all -# The infra-down cleanup happens automatically via defer +# Or with parallel execution +task test:parallel ``` ### Quick Validation @@ -181,15 +263,19 @@ task test:test-all ```bash # Fast smoke test task test:quick-test + +# Or parallel fast tests +task test:parallel-fast ``` -## Benefits of Modular Taskfile +## Benefits of Consolidated Test Namespace -1. **Separation of Concerns**: Testing infrastructure is isolated from build tasks -2. **Easier Maintenance**: Test-related tasks in one file -3. **Reusability**: Can be used standalone or included in root Taskfile -4. **Scalability**: Easy to add new test suites without cluttering root Taskfile -5. **Clarity**: Clear organization of infrastructure vs. test execution +1. **Single Namespace**: ALL tests accessible under `test:` +2. **Clear Hierarchy**: Unit → Integration → Acceptance → All +3. **Fast Iteration**: Unit tests run instantly, no infrastructure needed +4. **Easy Discovery**: `task --list | grep test:` shows all testing tasks +5. **Consistent Naming**: Predictable patterns for all test tasks +6. **Separation of Concerns**: Testing isolated from build tasks in root Taskfile ## Related Documentation diff --git a/testing/Taskfile.yml b/testing/Taskfile.yml index 0160d222f..571b486f3 100644 --- a/testing/Taskfile.yml +++ b/testing/Taskfile.yml @@ -1,12 +1,17 @@ version: '3' -# Integration Testing Infrastructure for Prism -# This Taskfile manages docker-compose lifecycle and integration test execution +# Prism Testing Infrastructure +# Consolidates ALL test tasks under the test: namespace +# - Unit tests (fast, no external dependencies) +# - Integration tests (requires docker-compose infrastructure) +# - Acceptance tests (pattern-level validation with testcontainers) +# - Parallel test execution # # Usage: -# task --taskfile testing/Taskfile.yml infra-up -# task --taskfile testing/Taskfile.yml test-all -# task --taskfile testing/Taskfile.yml infra-down +# task test:unit-all # Run all unit tests +# task test:integration-all # Run all integration tests +# task test:acceptance-all # Run all acceptance tests +# task test:all # Run everything vars: # Docker compose setup @@ -27,6 +32,136 @@ env: DOCKER_HOST: '{{.DOCKER_HOST}}' tasks: + # ============================================================================ + # Unit Tests (Fast, No External Dependencies) + # ============================================================================ + + unit-proxy: + desc: Run Rust proxy unit tests + sources: + - prism-proxy/src/**/*.rs + - prism-proxy/Cargo.toml + cmds: + - echo "Running Rust proxy unit tests..." + - cd prism-proxy && cargo test --lib + - echo "✓ Proxy unit tests passed" + + unit-core: + desc: Run Core SDK unit tests + sources: + - pkg/plugin/**/*.go + cmds: + - echo "Running Core SDK unit tests..." + - cd pkg/plugin && go test -v -cover ./... + - echo "✓ Core SDK unit tests passed" + + unit-prismctl: + desc: Run prismctl unit tests + sources: + - cmd/prismctl/**/*.go + cmds: + - echo "Running prismctl unit tests..." + - cd cmd/prismctl && go test -v -cover ./... + - echo "✓ prismctl unit tests passed" + + unit-operator: + desc: Run Kubernetes operator controller unit tests + sources: + - prism-operator/controllers/**/*.go + - prism-operator/api/**/*.go + cmds: + - echo "Running operator controller unit tests..." + - cd prism-operator/controllers && go test -v -cover ./... + - echo "✓ Operator controller unit tests passed" + + unit-memstore: + desc: Run MemStore driver unit tests + sources: + - pkg/drivers/memstore/**/*.go + cmds: + - echo "Running memstore unit tests..." + - cd pkg/drivers/memstore && go test -v -cover ./... + - echo "✓ memstore unit tests passed" + + unit-redis: + desc: Run Redis driver unit tests + sources: + - pkg/drivers/redis/**/*.go + cmds: + - echo "Running redis unit tests..." + - cd pkg/drivers/redis && go test -v -cover ./... + - echo "✓ redis unit tests passed" + + unit-nats: + desc: Run NATS driver unit tests + sources: + - pkg/drivers/nats/**/*.go + cmds: + - echo "Running nats unit tests..." + - cd pkg/drivers/nats && go test -v -cover ./... + - echo "✓ nats unit tests passed" + + unit-kafka: + desc: Run Kafka driver unit tests + sources: + - pkg/drivers/kafka/**/*.go + cmds: + - echo "Running kafka unit tests..." + - cd pkg/drivers/kafka && go test -v -cover ./... + - echo "✓ kafka unit tests passed" + + unit-postgres: + desc: Run Postgres driver unit tests + sources: + - pkg/drivers/postgres/**/*.go + cmds: + - echo "Running postgres unit tests..." + - cd pkg/drivers/postgres && go test -v -cover ./... + - echo "✓ postgres unit tests passed" + + unit-drivers: + desc: Run all driver unit tests + deps: [unit-memstore, unit-redis, unit-nats, unit-kafka, unit-postgres] + + unit-all: + desc: Run all unit tests (fast) + deps: [unit-proxy, unit-core, unit-prismctl, unit-operator, unit-drivers] + cmds: + - echo "✓ All unit tests passed" + + # ============================================================================ + # Parallel Test Execution + # ============================================================================ + + parallel: + desc: Run all tests in parallel + cmds: + - echo "Running tests in parallel..." + - uv run tooling/parallel_test.py --log-dir {{.TEST_LOGS_DIR}} + - echo "✓ Parallel tests complete" + + parallel-fast: + desc: Run fast tests only in parallel (skip acceptance) + cmds: + - echo "Running fast tests in parallel..." + - uv run tooling/parallel_test.py --fast --log-dir {{.TEST_LOGS_DIR}} + - echo "✓ Fast parallel tests complete" + + parallel-fail-fast: + desc: Run tests in parallel with fail-fast + cmds: + - echo "Running tests in parallel with fail-fast..." + - uv run tooling/parallel_test.py --fail-fast --log-dir {{.TEST_LOGS_DIR}} + - echo "✓ Parallel tests complete" + + acceptance-parallel: + desc: Run acceptance tests in parallel with matrix report + cmds: + - echo "Running acceptance tests in parallel..." + - mkdir -p {{.BUILD_DIR}}/reports + - uv run tooling/parallel_acceptance_test.py + - echo "✓ Parallel acceptance tests complete" + # ============================================================================ # Infrastructure Management # ============================================================================ @@ -251,27 +386,128 @@ tasks: - task: test-acceptance-unified - defer: { task: infra-down } + # ============================================================================ + # Testcontainers-Based Acceptance Tests + # ============================================================================ + + acceptance-testcontainers: + desc: Run all acceptance tests with testcontainers + deps: + - acceptance-testcontainers-interfaces + - acceptance-testcontainers-redis + - acceptance-testcontainers-nats + - acceptance-testcontainers-kafka + - acceptance-testcontainers-postgres + cmds: + - echo "✓ All testcontainers acceptance tests passed" + + acceptance-testcontainers-interfaces: + desc: Run interface-based acceptance tests with testcontainers + sources: + - tests/acceptance/interfaces/**/*.go + cmds: + - echo "Running interface-based acceptance tests..." + - cd tests/acceptance/interfaces && go test -v -timeout 10m ./... + - echo "✓ Interface-based acceptance tests passed" + + acceptance-testcontainers-redis: + desc: Run Redis acceptance tests with testcontainers + sources: + - tests/acceptance/redis/**/*.go + cmds: + - echo "Running Redis acceptance tests..." + - cd tests/acceptance/redis && go test -v -timeout 10m ./... + - echo "✓ Redis acceptance tests passed" + + acceptance-testcontainers-nats: + desc: Run NATS acceptance tests with testcontainers + sources: + - tests/acceptance/nats/**/*.go + cmds: + - echo "Running NATS acceptance tests..." + - cd tests/acceptance/nats && go test -v -timeout 10m ./... + - echo "✓ NATS acceptance tests passed" + + acceptance-testcontainers-kafka: + desc: Run Kafka acceptance tests with testcontainers + sources: + - tests/acceptance/kafka/**/*.go + cmds: + - echo "Running Kafka acceptance tests..." + - cd tests/acceptance/kafka && go test -v -timeout 10m ./... + - echo "✓ Kafka acceptance tests passed" + + acceptance-testcontainers-postgres: + desc: Run Postgres acceptance tests with testcontainers + sources: + - tests/acceptance/postgres/**/*.go + cmds: + - echo "Running Postgres acceptance tests..." + - cd tests/acceptance/postgres && go test -v -timeout 10m ./... + - echo "✓ Postgres acceptance tests passed" + + acceptance-testcontainers-quiet: + desc: Run all testcontainers acceptance tests in quiet mode + cmds: + - echo "Running acceptance tests in quiet mode..." + - PRISM_TEST_QUIET=1 task acceptance-testcontainers + - echo "✓ All acceptance tests passed (quiet mode)" + + # ============================================================================ + # Additional Integration Tests + # ============================================================================ + + integration-rust: + desc: Run Rust integration tests (requires built binaries) + sources: + - prism-proxy/tests/**/*.rs + cmds: + - echo "Running Rust integration tests..." + - cd prism-proxy && cargo test --test integration_test -- --ignored --nocapture + - echo "✓ Rust integration tests passed" + + integration-k8s: + desc: Run Kubernetes integration tests (requires local K8s cluster and images) + sources: + - tests/integration/k8s/**/*.go + - prism-operator/api/**/*.go + - prism-operator/controllers/**/*.go + cmds: + - echo "Running Kubernetes integration tests..." + - echo "Prerequisites - K8s cluster running and images built" + - cd tests/integration/k8s && go test -v -timeout 30m ./... + - echo "✓ Kubernetes integration tests passed" + + integration-k8s-short: + desc: Run quick Kubernetes integration tests (minimal test only) + cmds: + - echo "Running quick Kubernetes integration tests..." + - cd tests/integration/k8s && go test -v -timeout 10m -short ./... + - echo "✓ Quick Kubernetes integration tests passed" + # ============================================================================ # Comprehensive Test Suites # ============================================================================ - test-all: - desc: Run all integration and acceptance tests with infrastructure + all: + desc: Run ALL tests (unit + integration + acceptance + MCP) deps: [infra-up] cmds: - echo "Running comprehensive test suite..." - mkdir -p {{.TEST_LOGS_DIR}} + - task: unit-all - task: test-integration-all - task: test-acceptance-all - task: test-backends-all + - task: acceptance-testcontainers - defer: { task: infra-down } - echo "✓ All tests passed!" - test-all-with-mcp: + all-with-mcp: desc: Run all tests including MCP integration tests cmds: - echo "Running complete test suite with MCP..." - - task: test-all + - task: all - task: test-mcp-full - echo "✓ Complete test suite passed!" @@ -303,25 +539,35 @@ tasks: help: desc: Show available testing tasks cmds: - - echo "Integration Testing Infrastructure" + - echo "Prism Testing Infrastructure (test namespace)" + - echo "" + - echo "Unit Tests (Fast, No Dependencies)" + - echo " unit-all - Run all unit tests" + - echo " unit-proxy - Run Rust proxy unit tests" + - echo " unit-core - Run Core SDK unit tests" + - echo " unit-drivers - Run all driver unit tests" + - echo "" + - echo "Integration Tests (Requires Infrastructure)" + - echo " test-integration-all - Run all integration tests" + - echo " integration-rust - Run Rust integration tests" + - echo " integration-k8s - Run Kubernetes integration tests" + - echo "" + - echo "Acceptance Tests" + - echo " test-acceptance-all - Run all acceptance tests (docker-compose)" + - echo " acceptance-testcontainers - Run all acceptance tests (testcontainers)" - echo "" - echo "Infrastructure Management" - echo " infra-up - Start full test infrastructure" - - echo " infra-down - Stop full test infrastructure" + - echo " infra-down - Stop test infrastructure" - echo " infra-status - Check infrastructure status" - - echo " infra-logs - View all infrastructure logs" - - echo " mcp-infra-up - Start MCP test infrastructure" - - echo " mcp-infra-down - Stop MCP test infrastructure" - echo "" - - echo "Integration Tests" - - echo " test-integration-all - Run all integration tests" - - echo " test-acceptance-all - Run all acceptance tests" - - echo " test-backends-all - Run all backend tests" - - echo " test-mcp-full - Run MCP integration tests" + - echo "Parallel Execution" + - echo " parallel - Run all tests in parallel" + - echo " parallel-fast - Run fast tests in parallel" - echo "" - echo "Comprehensive Suites" - - echo " test-all - Run all tests with infrastructure" - - echo " test-all-with-mcp - Run all tests including MCP" + - echo " all - Run ALL tests (unit + integration + acceptance)" + - echo " all-with-mcp - Run all tests including MCP" - echo " quick-test - Quick smoke test" - echo "" - - echo "For full list - task --taskfile testing/Taskfile.yml --list" + - echo "Full list - task --list | grep test"