[CAE-1072] Hitless Upgrades

[ndyakov]

Hitless Upgrades

Seamless Redis connection handoffs during topology changes without interrupting operations.

Quick Start

import "github.com/redis/go-redis/v9/hitless"

opt := &redis.Options{
    Addr:     "localhost:6379",
    Protocol: 3, // RESP3 required
    HitlessUpgrades: &redis.HitlessUpgradeConfig{
        Mode: hitless.MaintNotificationsEnabled, // or MaintNotificationsAuto
    },
}
client := redis.NewClient(opt)

Modes

• MaintNotificationsDisabled: Hitless upgrades are completely disabled
• MaintNotificationsEnabled: Hitless upgrades are forcefully enabled (fails if server doesn't support it)
• MaintNotificationsAuto: Hitless upgrades are enabled if server supports it (default)

Configuration

import "github.com/redis/go-redis/v9/hitless"

Config: &hitless.Config{
    Mode:                       hitless.MaintNotificationsAuto, // Notification mode
    MaxHandoffRetries:           3,  // Retry failed handoffs
    HandoffTimeout:             15 * time.Second, // Handoff operation timeout
    RelaxedTimeout:             10 * time.Second, // Extended timeout during migrations
    PostHandoffRelaxedDuration: 20 * time.Second, // Keep relaxed timeout after handoff
    LogLevel:                   1,  // 0=errors, 1=warnings, 2=info, 3=debug
    MaxWorkers:                 15, // Concurrent handoff workers
    HandoffQueueSize:           50, // Handoff request queue size
}

Worker Scaling

• Auto-calculated: min(10, PoolSize/3) - scales with pool size, capped at 10
• Explicit values: max(10, set_value) - enforces minimum 10 workers
• On-demand: Workers created when needed, cleaned up when idle

Queue Sizing

• Auto-calculated: 10 × MaxWorkers, capped by pool size
• Always capped: Queue size never exceeds pool size

Metrics Hook Example

A metrics collection hook is available in example_hooks.go that demonstrates how to monitor hitless upgrade operations:

import "github.com/redis/go-redis/v9/hitless"

metricsHook := hitless.NewMetricsHook()
// Use with your monitoring system

The metrics hook tracks:

• Handoff success/failure rates
• Handoff duration
• Queue depth
• Worker utilization
• Connection lifecycle events

Requirements

• RESP3 Protocol: Required for push notifications

[Copilot]

Pull Request Overview

This PR implements hitless upgrades for Redis connections, enabling seamless connection handoffs during topology changes without interrupting operations. It adds comprehensive support for RESP3 push notifications to handle cluster migration events gracefully.

Key changes include:

• Hitless upgrade manager and pool hooks for event-driven connection handoffs
• Enhanced connection pool with relaxed timeout support and atomic state management
• RESP3 push notification handling for cluster migration events (MOVING, MIGRATING, etc.)

Reviewed Changes

Copilot reviewed 46 out of 48 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
universal.go	Added HitlessUpgradeConfig support to universal options
redis.go	Integrated hitless manager and enhanced connection initialization
options.go	Added hitless configuration with endpoint auto-detection
internal/pool/conn.go	Enhanced connection with atomic handoff state and relaxed timeouts
internal/pool/pool.go	Added pool hooks system and improved connection lifecycle management
hitless/pool_hook.go	Implemented event-driven connection handoff processing
hitless/notification_handler.go	Added RESP3 push notification handlers for cluster events

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[ofekshenawa]

When MOVING points to a different host/FQDN or an IP, we reuse the original TLS config (including ServerName) while dialing the new endpoint. That can cause hostname verification mismatches (e.g., cert valid for old host, not new host/IP).

[ndyakov]

Correct. Since there are no certificates transmitted as part of the notification, I am assuming there would be a root level configuration for tls which should cover the new endpoint.