Add unified scanner with TensorTrap + ClamAV integration

[JustMaier]

Summary

• Adds unified scanner container combining TensorTrap ML security scanner with ClamAV antivirus
• Implements shadow mode to run both legacy (picklescan) and unified scanners in parallel for safe rollout
• Adds metrics endpoint to track scanner agreement rates before migration
• Includes e2e test infrastructure for local development

Key Changes

New Components

• unified-scanner: Docker container with TensorTrap + ClamAV
• clamav-updater: Sidecar container for automatic virus definition updates
• ShadowModeMetrics: Tracks comparison between legacy and unified scanner results

Features

• Shadow mode runs both scanners in parallel, uses legacy results but logs discrepancies
• /metrics/shadow endpoint shows agreement rate, discrepancies, and recommendations
• Docker-in-Docker support with shared volumes for file access
• Path validation to prevent command injection
• Atomic file writes for metrics persistence

Performance

• TensorTrap scans 2GB .ckpt files in ~3.5 seconds vs 4-5 minutes with legacy picklescan
• Uses optimized fork: JustMaier/TensorTrap

Test plan

• Shadow mode test with 2GB .ckpt file - both scanners agree (100% agreement rate)
• Metrics endpoint returns correct statistics
• E2E test infrastructure working
• Test with known malicious samples to verify discrepancy detection

Configuration

Enable shadow mode:

ScannerOptions__ShadowMode=true

Enable unified scanner only (after shadow mode validation):

ScannerOptions__UseUnifiedScanner=true
ScannerOptions__ShadowMode=false

🤖 Generated with Claude Code