feat(api): add cache

[hspedro]

🚀 Feature: Caching System for Translation and Detection Services

Overview

This PR introduces a robust caching system for Babeltron's translation and language detection services. The implementation significantly improves response times for repeated requests by storing previous results in a Valkey (Redis-compatible) cache with configurable TTL.

Key Features

1. Cache Infrastructure

• Cache Interface: Created an abstract base class (CacheInterface) that defines the contract for all cache implementations with save, get, and delete methods.
• Valkey Implementation: Implemented a Valkey-based cache client with:
  • Singleton pattern to ensure a single connection per application instance
  • Comprehensive error handling via decorators
  • Automatic serialization/deserialization of complex data types
  • Memory usage optimization (configured to use max 512MB with LRU eviction policy)

2. Cache Service Layer

• Generic Cache Service: Implemented a type-safe cache service with specialized methods for translation and detection results
• Intelligent Key Generation: Created utility functions to generate consistent cache keys based on:
  • Operation type (translate/detect)
  • Input text (normalized and sanitized)
  • Source and target languages (for translations)

3. API Integration

• Translation Endpoint: Updated to check cache before performing translation
• Detection Endpoint: Updated to check cache before performing language detection
• Cache Flags: Added cached: boolean field to responses to indicate cache hits
• Telemetry: Added cache hit/miss tracking to OpenTelemetry spans

4. Configuration

• Added environment variables for cache configuration:
  • CACHE_HOST: Hostname for the cache server
  • CACHE_PORT: Port for the cache server
  • CACHE_TTL_SECONDS: Time-to-live for cached items (default: 3600s/1h)

5. Infrastructure Updates

• Added Valkey service to Docker Compose configuration
• Updated OpenTelemetry collector configuration to use OTLP exporter instead of deprecated Jaeger exporter
• Fixed NumPy compatibility issues by pinning to version <2.0.0

6. Testing

• Added comprehensive unit tests for:
  • Cache interface implementation
  • Cache service layer
  • Key generation utilities
  • Integration with translation and detection endpoints

Technical Details

Cache Key Format

• Translation: translate:{source_lang}:{target_lang}:{normalized_text}
• Detection: detect:{normalized_text}

Text normalization includes:

• Unicode normalization (NFC form)
• Whitespace trimming
• Lowercase conversion

Performance Considerations

• The cache is configured with a memory limit of 512MB
• LRU (Least Recently Used) eviction policy ensures optimal memory usage
• Default TTL of 1 hour balances freshness with performance
• JSON serialization for complex data types ensures compatibility

Breaking Changes

None. This is a non-breaking enhancement that maintains backward compatibility with existing API contracts.

Future Improvements

• Add cache statistics endpoint for monitoring
• Implement cache warming for common translations
• Add support for additional cache backends (Memcached, Redis Cluster, etc.)
• Implement cache invalidation strategies for model updates

Dependencies

• Added valkey package (v6.1.0) - Redis-compatible Python client
• Fixed NumPy compatibility by pinning to version <2.0.0

Testing Instructions

1. Start the application with Docker Compose:

   docker-compose up -d

2. Make a translation request:

   curl -X POST "http://localhost:8000/api/v1/translate" \
     -H "Content-Type: application/json" \
     -d '{"text": "Hello world", "source_language": "en", "target_language": "fr"}'

3. Make the same request again and observe the cached: true flag in the response and faster response time.

4. Similarly, test the detection endpoint:

   curl -X POST "http://localhost:8000/api/v1/detect" \
     -H "Content-Type: application/json" \
     -d '{"text": "Hello world"}'