HTTP Collection API (#7)

[prosdev]

Implements Issue #7: HTTP Collection API with health checks, dependency injection, and CI integration.

Summary

Built a production-ready HTTP collection API with FastAPI that:

• Accepts events at multiple endpoints (core + Segment-compatible)
• Always returns 202 Accepted (no edge rejections)
• Routes events through EventQueue (queue-agnostic design)
• Implements proper health checks (liveness vs readiness)
• Verifies external dependencies (Firestore connectivity)

What's Implemented

Collection Endpoints

• POST /collect and POST /collect/{stream} - Accept any JSON payload
• Always returns 202 Accepted (invalid events → ErrorStore)

Convenience Endpoints (Segment-compatible)

• POST /v1/identify → routes to "users" stream
• POST /v1/track → routes to "events" stream
• POST /v1/page → routes to "pages" stream

Health Checks

• GET /health - Liveness check (returns 200 if process running)
• GET /ready - Readiness check (verifies Firestore connectivity, returns 503 if unhealthy)

Architecture

• Dependency Injection:

  • get_settings() - Singleton settings from environment
  • get_queue() - Wires entire pipeline (adapter → sequencer → buffer → processor → queue)
  • get_event_store() - EventStore instance for health checks

• FastAPI Application:

  • Lifespan manager handles queue lifecycle
  • Startup: await queue.start() (start workers, buffer flusher)
  • Shutdown: await queue.stop() (drain queue, flush buffers)

• Queue-Agnostic Design:

  • API depends on EventQueue protocol (not DirectQueue/AsyncQueue)
  • Can swap queue modes via EVENTKIT_QUEUE_MODE config
  • Works with DirectQueue, AsyncQueue, future PubSubQueue

Storage Layer Updates

• Added health_check() method to EventStore protocol
• Implemented health_check() in FirestoreEventStore
• Performs lightweight connectivity check (get non-existent document)

Tests

14 new API tests (tests/unit/api/test_router.py):

• Collection endpoint tests (single events, stream routing, payload acceptance)
• Convenience endpoint tests (correct stream routing)
• Health check tests (liveness and readiness, including unhealthy state)

Coverage: 94% for API layer, 150 total tests passing (unit + integration)

CI/CD

Updated .github/workflows/test.yml to use docker-compose (consistent with local dev):

• docker-compose up -d --wait leverages healthchecks from docker-compose.yml
• Firestore emulator runs as a service
• All tests (unit + integration) run against emulator
• Industry standard approach (same commands locally and in CI)

Documentation

LOCAL_DEV.md

Comprehensive local development guide:

• Docker Compose setup for Firestore emulator
• Running FastAPI server
• Manual testing with curl examples

README.md

Updated with link to LOCAL_DEV.md for quick onboarding

CLAUDE.md

Updated with latest patterns:

• Queue-agnostic processor architecture
• API & dependency injection examples
• Health checks (liveness vs readiness)
• Docker Compose workflow

Manual Testing

All functionality verified with manual testing:

• Health checks work correctly (liveness always 200, readiness 200/503)
• Collection endpoints accept events and return 202
• Stream routing works (default, custom, convenience endpoints)
• Valid events processed and stored in Firestore (6 events across 4 streams)
• Invalid events routed to ErrorStore (1 error captured)
• Buffer flushing works (events appear after 5-second timeout)
• Firestore failure handled gracefully (503 on /ready)

Files Changed

New Files:

• src/eventkit/api/__init__.py
• src/eventkit/api/app.py
• src/eventkit/api/dependencies.py
• src/eventkit/api/router.py
• tests/unit/api/__init__.py
• tests/unit/api/test_router.py
• LOCAL_DEV.md

Updated Files:

• src/eventkit/stores/event_store.py (added health_check protocol)
• src/eventkit/stores/firestore.py (implemented health_check)
• src/eventkit/config.py (removed unused FIRESTORE_*_COLLECTION settings)
• .github/workflows/test.yml (docker-compose integration)
• README.md (added LOCAL_DEV.md link)
• CLAUDE.md (updated patterns)
• specs/core-pipeline/tasks.md (marked Task 10 complete)

Closes

• Closes HTTP API #7