Multipath QUIC does not provide resolver-level failover under path failure

[anonuser201]

Slipstream supports multiple resolvers and uses QUIC multipath. However, in practice, resolver-level failover does not occur when one resolver becomes unhealthy.

Reproduction:

slipstream-client \
  --tcp-listen-port 2052 \
  --resolver 1.1.1.1:53 \
  --resolver 8.8.8.8:53 \
  --domain ns.domain.tld

Observed behavior:
When the first resolver is consistently unhealthy, the client repeatedly retries the same resolver with exponential backoff:

 INFO Listening on TCP port 2052 (host ::)
 WARN Connection closed event=close state=picoquic_state_disconnected local_error=0x433 remote_error=0x0 local_app=0x0 remote_app=0x0 ready=false
 WARN Connection closed; reconnecting in 250ms
 WARN Path for resolver [::ffff:1.1.1.1]:53 became unavailable; resetting state
 WARN Connection closed event=close state=picoquic_state_disconnected local_error=0x433 remote_error=0x0 local_app=0x0 remote_app=0x0 ready=false
 WARN Connection closed; reconnecting in 500ms
 WARN Path for resolver [::ffff:1.1.1.1]:53 became unavailable; resetting state
 WARN Connection closed event=close state=picoquic_state_disconnected local_error=0x433 remote_error=0x0 local_app=0x0 remote_app=0x0 ready=false
 WARN Connection closed; reconnecting in 1000ms
 WARN Path for resolver [::ffff:1.1.1.1]:53 became unavailable; resetting state
 WARN Connection closed event=close state=picoquic_state_disconnected local_error=0x433 remote_error=0x0 local_app=0x0 remote_app=0x0 ready=false
 WARN Connection closed; reconnecting in 2000ms
 WARN Path for resolver [::ffff:1.1.1.1]:53 became unavailable; resetting state
(repeats)

The client never switches to the second resolver, even though it is configured and available.

slipstream-client \
  --tcp-listen-port 2052 \
  --resolver 8.8.8.8:53 \
  --resolver 1.1.1.1:53 \
  --domain ns.domain.tld

 INFO Listening on TCP port 2052 (host ::)
 INFO Connection ready
 INFO Added path [::ffff:1.1.1.1]:53
 WARN Path for resolver [::ffff:1.1.1.1]:53 became unavailable; resetting state
 WARN Failed adding path [::ffff:1.1.1.1]:53 (attempt 1), retrying in 250ms
 WARN Failed adding path [::ffff:1.1.1.1]:53 (attempt 2), retrying in 500ms
 INFO Added path [::ffff:1.1.1.1]:53
 WARN Path for resolver [::ffff:1.1.1.1]:53 became unavailable; resetting state
 WARN Failed adding path [::ffff:1.1.1.1]:53 (attempt 1), retrying in 250ms
 WARN Failed adding path [::ffff:1.1.1.1]:53 (attempt 2), retrying in 500ms
 INFO Added path [::ffff:1.1.1.1]:53
 WARN Path for resolver [::ffff:1.1.1.1]:53 became unavailable; resetting state
 WARN Failed adding path [::ffff:1.1.1.1]:53 (attempt 1), retrying in 250ms
 WARN Failed adding path [::ffff:1.1.1.1]:53 (attempt 2), retrying in 500ms
 INFO Added path [::ffff:1.1.1.1]:53
(repeats)

Expected behavior:
Once a resolver path is marked unavailable, the client should:

• deprioritize or exclude it, and
• attempt another configured resolver.

Notes:
This suggests that while multipath exists at the QUIC transport level, there is no resolver selection or failover policy at the client level. Multipath capability alone does not guarantee reliability without path health semantics (e.g., unhealthy exclusion, backoff with rotation).

[Mygod]

On top of my mind, the first resolver needs to be available on startup for the handshake. Multipath is only active after the initial handshake. Do you still observe this issue after connection established?

[anonuser201]

Here are additional tests and observations after your question about handshake vs multipath activation.

Test setup

Command used:

slipstream-client \
  --tcp-listen-port 2052 \
  --resolver 8.8.8.8:53 \
  --resolver 1.1.1.1:53 \
  --resolver 9.9.9.9:53 \
  --domain ns.domain.tld

To simulate a resolver becoming unhealthy after the connection is already established, I used Linux routing:

Make the first resolver unreachable:

sudo ip route add unreachable 8.8.8.8/32

Restore it:

sudo ip route del unreachable 8.8.8.8/32

---

Test 1 – Initial resolver healthy (baseline)

• 8.8.8.8 is healthy at startup.
• Connection is established successfully.
• Multipath paths are added for the other resolvers.

This confirms that:

• Initial handshake works.
• Multipath becomes active after connection establishment.

---

Test 2 – First resolver becomes unhealthy after handshake

Steps:

1. Start with 8.8.8.8 healthy → connection established.

2. Make 8.8.8.8 unreachable using:

   sudo ip route add unreachable 8.8.8.8/32

3. Observe client behavior.

Observed behavior (WARN only, repeated):

• After 8.8.8.8 becomes unreachable, the client enters a retry loop.
• The logs show repeated retry attempts for other configured resolvers (e.g. 1.1.1.1 and 9.9.9.9), with exponential backoff.
• However, despite retries being logged, a new stable connection is not consistently established over another resolver.

Additional observation:

• In some runs, when the initial connection over 8.8.8.8 is already established, the connection may remain partially functional even after 8.8.8.8 becomes unreachable.
• In those cases, the client does not reliably promote another resolver as an active path, and retry logs for other resolvers may stop appearing.
• This behavior appears non-deterministic, but in all observed cases, resolver-level failover to a healthy resolver is not reliably achieved.

In other words:

• Multipath is active.
• The failure happens after handshake.
• Resolver-level failover is inconsistent and does not reliably switch the active connection to another healthy resolver.

When 8.8.8.8 is made reachable again:

sudo ip route del unreachable 8.8.8.8/32

the connection recovers.

---

Test 3 – Healthy resolver works when it is first

To rule out resolver-specific issues:

Command:

slipstream-client \
  --tcp-listen-port 2052 \
  --resolver 9.9.9.9:53 \
  --resolver 1.1.1.1:53 \
  --resolver 8.8.8.8:53 \
  --domain ns.domain.tld

Result:

• Connection is established successfully.
• Confirms that 9.9.9.9 itself is healthy and works fine when used as the initial resolver.