test: Update and expand E2E test coverage

[Defilan]

Summary

E2E tests need updates to cover current functionality and close gaps. Several tests are outdated, hardcoded to GKE, or have incomplete implementations.

Current Issues

e2e_test.go

• TODO at lines 268-276 for CR reconciliation tests never implemented
• Should test Model and InferenceService CR lifecycle

catalog_e2e_test.go

• Deploy tests skipped (lines 119, 136)
• Missing tests for new 32B models (qwen-2.5-32b, qwen-2.5-coder-32b, qwen-3-32b)
• No tests for benchmark command
• No tests for cache command

gpu_test.sh

• Uses Q8_0 quantization - doesn't match catalog (Q5_K_M)
• Hardcoded GKE node selector gpu-pool (line 168)
• Assumes monitoring namespace exists
• Single-GPU only - no multi-GPU coverage

multi-gpu-test-plan.md

• Comprehensive plan exists but no automated implementation
• Manual checklist only, not executable tests

Missing Test Coverage

Area	Current State	Priority
benchmark command	No tests	High
cache command	No tests	Medium
Multi-GPU scenarios	Manual plan only	High
32B catalog models	Not tested	Medium
Metal/macOS	No tests	Low
CR reconciliation	TODO placeholder	Medium

Proposed Improvements

1. Update gpu_test.sh to use catalog models and be platform-agnostic
2. Add benchmark E2E test - critical path for users
3. Add cache E2E test - PVC operations
4. Implement multi-GPU E2E from existing test plan
5. Add 32B model tests to catalog_e2e_test.go
6. Complete CR reconciliation tests in e2e_test.go

Related

• Add automated E2E tests for multi-GPU scenarios #48 - Add automated E2E tests for multi-GPU scenarios
• test: Improve unit test coverage across packages #89 - Improve unit test coverage