Deadline is parsed but not enforced — handler runs to completion after Connect-Timeout-Ms lapses

[iainmcgin]

The runtime parses Connect-Timeout-Ms / grpc-timeout headers into RequestContext::deadline: Option<Instant>, but does not cancel the handler future when the deadline lapses. The handler must check ctx.deadline itself.

This differs from tonic and connect-go, both of which cancel the handler when the deadline passes (tonic drops the future; connect-go's context.Context is cancelled). A user who writes a slow handler and relies on the protocol timeout for protection will find their handler running to completion, holding DB connections, locks, and worker-pool slots, only for the response to be discarded because the client has already gone away. Under load this turns a slow downstream into a cascading resource exhaustion.

It's a defensible design choice — a handler may legitimately want to commit a transaction it has started even if the client gave up — but it's a surprising default and should at minimum be a documented, explicit choice.

Proposed direction

• Default to enforcing the deadline: wrap the handler future in tokio::time::timeout(deadline - now, handler) and translate the timeout into a DeadlineExceeded error. This matches the behavior of every other framework in the comparison set and the gRPC spec's intent.
• Add an opt-out: Server::enforce_deadlines(false) (and an equivalent for the axum/tower integration path, perhaps a RouterConfig knob) for services that need to outlive the caller.
• Document the behavior and the opt-out in the guide's "Production hardening" section.
• Consider exposing the remaining time (ctx.time_remaining() -> Option<Duration>) so handlers can budget downstream calls, similar to gRPC-Go's ctx.Deadline() idiom.

Open questions

• For client/bidi streaming handlers, should the deadline apply to the whole stream lifetime or be resettable per message?
• Whether cancellation should be "drop the future" (Rust idiom, may leak partial work) or "signal and grace period" (closer to Go).

Scope

Medium — touches the dispatcher hot path, needs careful testing against the conformance suite's timeout cases.

[tejas-dharani]

Wanted to flag what I found running this down — the unary path may already be enforced; the gap looks specific to streaming.

Reading service.rs:1575-1584 and the streaming dispatch sites at 2073, 2210, 2455, the dispatcher wraps every handler future in tokio::time::timeout. Verified end-to-end: connect-timeout-ms: 200 sent as a raw header (no client-side timeout, so only the server can cancel) returns DeadlineExceeded with "request timeout" at ~204ms, and the handler's completion flag stays false.

For streaming, the timeout wraps the handler-construction future — which resolves the moment the handler returns its Stream value. StreamingResponseBody then drains the stream with no deadline awareness: a 5-item stream emitting at 500ms intervals ran to 2500ms despite a 200ms client deadline. Same gap on the inbound side of client-streaming/bidi — requests.next().await blocks indefinitely on a stalled client.

Flagging in case it sharpens the fix scope.