client: stream call_client_stream request body via channel#84
Merged
iainmcgin merged 1 commit intoanthropics:mainfrom May 7, 2026
Merged
client: stream call_client_stream request body via channel#84iainmcgin merged 1 commit intoanthropics:mainfrom
iainmcgin merged 1 commit intoanthropics:mainfrom
Conversation
|
All contributors have signed the CLA ✍️ ✅ |
iainmcgin
approved these changes
May 7, 2026
…s#84) Previously call_client_stream concatenated every encoded envelope into a single BytesMut and handed it to full_body(), so the transport never saw the first byte until the entire iterator had been drained. Peak memory was O(total) and a slow- or large-yielding iterator buffered everything in process memory before any of it hit the wire. Switch to the same channel-backed body pattern call_bidi_stream uses: a depth-32 mpsc channel feeds ChannelBody, the iterator drain pushes one envelope per item with natural backpressure on HTTP/2 flow control, and the response future is driven concurrently so frames flush as they're produced. Peak memory drops to roughly channel_depth * envelope_size. The transport send is dispatched onto a separate task so its body is polled while we drain the iterator (a transport whose send() future contains the actual I/O would otherwise deadlock once the channel filled). On wasm32 there is no tokio runtime; spawn onto wasm-bindgen-futures' single-threaded executor instead. Either way, the response is bridged back via a oneshot so the awaitee is uniform across architectures. The IntoIterator<Item = Req> public signature is unchanged, so codegen and existing callers are unaffected. A new regression test in tests/streaming installs a recording ClientTransport into the generated EchoServiceClient and asserts the body arrives as N frames for N messages instead of one concatenated frame; this fails on the previous buffered code (1 frame of 68 bytes) and passes on the new code.
0c80c70 to
41e39d1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously call_client_stream concatenated every encoded envelope into a single BytesMut and handed it to full_body(), so the transport never saw the first byte until the entire iterator had been drained. Peak memory was O(total) and a slow- or large-yielding iterator buffered everything in process memory before any of it hit the wire.
Switch to the same channel-backed body pattern call_bidi_stream uses: a depth-32 mpsc channel feeds ChannelBody, the iterator drain pushes one envelope per item with natural backpressure on HTTP/2 flow control, and the response future is driven concurrently so frames flush as they're produced. Peak memory drops to roughly channel_depth * envelope_size.
The transport send is dispatched onto a separate task so its body is polled while we drain the iterator (a transport whose send() future contains the actual I/O would otherwise deadlock once the channel filled). On wasm32 there is no tokio runtime; spawn onto wasm-bindgen-futures' single-threaded executor instead. Either way, the response is bridged back via a oneshot so the awaitee is uniform across architectures.
The IntoIterator<Item = Req> public signature is unchanged, so codegen and existing callers are unaffected. A new regression test in tests/streaming installs a recording ClientTransport into the generated EchoServiceClient and asserts the body arrives as N frames for N messages instead of one concatenated frame; this fails on the previous buffered code (1 frame of 68 bytes) and passes on the new code.