-
Notifications
You must be signed in to change notification settings - Fork 935
Stream retry support part 2: Introduce a new split method in AsyncRequestBody that returns an SdkP… #6346
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Stream retry support part 2: Introduce a new split method in AsyncRequestBody that returns an SdkP… #6346
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
d94cec4
Introduce a new split method in AsyncRequestBody that returns an SdkP…
zoewangg 3c91cc9
Fix build and add more tests
zoewangg 004733b
Refactoring based on API surface are review
zoewangg 0c2000e
Add more tests and address comments
zoewangg b61019e
Deprecate split, update documentation and add changelog entries
zoewangg File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
6 changes: 6 additions & 0 deletions
6
.changes/next-release/deprecation-AWSSDKforJavav2-6069419.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
{ | ||
"type": "deprecation", | ||
"category": "AWS SDK for Java v2", | ||
"contributor": "", | ||
"description": "Deprecate `AsyncRequestBody#split` in favor of `AsyncRequestBody#splitCloseable` that takes the same input but returns `SdkPublisher<CloseableAsyncRequestBody>`" | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
{ | ||
"type": "feature", | ||
"category": "AWS SDK for Java v2", | ||
"contributor": "", | ||
"description": "Add `AsyncRequestBody#splitCloseable` API that returns a Publisher of `ClosableAsyncRequestBody`" | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
{ | ||
"type": "feature", | ||
"category": "AWS SDK for Java v2", | ||
"contributor": "", | ||
"description": "Introduce CloseableAsyncRequestBody interface that extends both AsyncRequestBody and SdkAutoClosable interfaces" | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
{ | ||
"type": "feature", | ||
"category": "AWS SDK for Java v2", | ||
"contributor": "", | ||
"description": "Introduce BufferedSplittableAsyncRequestBody that enables splitting into retryable sub-request bodies." | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
113 changes: 113 additions & 0 deletions
113
...e/src/main/java/software/amazon/awssdk/core/async/BufferedSplittableAsyncRequestBody.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,113 @@ | ||
/* | ||
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"). | ||
* You may not use this file except in compliance with the License. | ||
* A copy of the License is located at | ||
* | ||
* http://aws.amazon.com/apache2.0 | ||
* | ||
* or in the "license" file accompanying this file. This file is distributed | ||
* on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either | ||
* express or implied. See the License for the specific language governing | ||
* permissions and limitations under the License. | ||
*/ | ||
|
||
package software.amazon.awssdk.core.async; | ||
|
||
import java.nio.ByteBuffer; | ||
import java.util.Optional; | ||
import org.reactivestreams.Subscriber; | ||
import software.amazon.awssdk.annotations.SdkPublicApi; | ||
import software.amazon.awssdk.core.internal.async.SplittingPublisher; | ||
import software.amazon.awssdk.utils.Validate; | ||
|
||
/** | ||
* An {@link AsyncRequestBody} decorator that enables splitting into retryable sub-request bodies. | ||
* | ||
* <p>This wrapper allows any {@link AsyncRequestBody} to be split into multiple parts where each part | ||
* can be retried independently. When split, each sub-body buffers its portion of data, enabling | ||
* resubscription if a retry is needed (e.g., due to network failures or service errors).</p> | ||
* | ||
* <p><b>Retry Requirements:</b></p> | ||
* <p>Retry is only possible if all the data has been successfully buffered during the first subscription. | ||
* If the first subscriber fails to consume all the data (e.g., due to early cancellation or errors), | ||
* subsequent retry attempts will fail since the complete data set is not available for resubscription.</p> | ||
* | ||
* <p><b>Usage Example:</b></p> | ||
* {@snippet : | ||
* AsyncRequestBody originalBody = AsyncRequestBody.fromString("Hello World"); | ||
* BufferedSplittableAsyncRequestBody retryableBody = | ||
* BufferedSplittableAsyncRequestBody.create(originalBody); | ||
* } | ||
* | ||
* <p><b>Performance Considerations:</b></p> | ||
* <p>This implementation buffers data in memory to enable retries, but memory usage is controlled by | ||
* the {@code bufferSizeInBytes} configuration. However, this buffering limits the ability to request | ||
* more data from the original AsyncRequestBody until buffered data is consumed (i.e., when subscribers | ||
* closes sub-body), which may increase latency compared to non-buffered implementations. | ||
* | ||
* @see AsyncRequestBody | ||
* @see AsyncRequestBodySplitConfiguration | ||
* @see CloseableAsyncRequestBody | ||
*/ | ||
@SdkPublicApi | ||
public final class BufferedSplittableAsyncRequestBody implements AsyncRequestBody { | ||
private final AsyncRequestBody delegate; | ||
|
||
private BufferedSplittableAsyncRequestBody(AsyncRequestBody delegate) { | ||
this.delegate = delegate; | ||
zoewangg marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
|
||
/** | ||
* Creates a new {@link BufferedSplittableAsyncRequestBody} that wraps the provided {@link AsyncRequestBody}. | ||
* | ||
* @param delegate the {@link AsyncRequestBody} to wrap and make retryable. Must not be null. | ||
* @return a new {@link BufferedSplittableAsyncRequestBody} instance | ||
* @throws NullPointerException if delegate is null | ||
*/ | ||
public static BufferedSplittableAsyncRequestBody create(AsyncRequestBody delegate) { | ||
Validate.paramNotNull(delegate, "delegate"); | ||
return new BufferedSplittableAsyncRequestBody(delegate); | ||
} | ||
|
||
@Override | ||
public Optional<Long> contentLength() { | ||
return delegate.contentLength(); | ||
} | ||
|
||
/** | ||
* Splits this request body into multiple retryable parts based on the provided configuration. | ||
* | ||
* <p>Each part returned by the publisher will be a {@link CloseableAsyncRequestBody} that buffers | ||
* its portion of data, enabling resubscription for retry scenarios. This is the key difference from non-buffered splitting - | ||
* each part can be safely retried without data loss. | ||
* | ||
* <p>The splitting process respects the chunk size and buffer size specified in the configuration | ||
* to optimize memory usage. | ||
* | ||
* <p>The subscriber MUST close each {@link CloseableAsyncRequestBody} to ensure resource is released | ||
* | ||
* @param splitConfiguration configuration specifying how to split the request body | ||
* @return a publisher that emits retryable closable request body parts | ||
* @see AsyncRequestBodySplitConfiguration | ||
*/ | ||
@Override | ||
public SdkPublisher<CloseableAsyncRequestBody> splitCloseable(AsyncRequestBodySplitConfiguration splitConfiguration) { | ||
return SplittingPublisher.builder() | ||
.asyncRequestBody(this) | ||
.splitConfiguration(splitConfiguration) | ||
.retryableSubAsyncRequestBodyEnabled(true) | ||
.build(); | ||
} | ||
|
||
@Override | ||
public void subscribe(Subscriber<? super ByteBuffer> s) { | ||
delegate.subscribe(s); | ||
} | ||
|
||
@Override | ||
public String body() { | ||
return delegate.body(); | ||
} | ||
} |
26 changes: 26 additions & 0 deletions
26
core/sdk-core/src/main/java/software/amazon/awssdk/core/async/CloseableAsyncRequestBody.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
/* | ||
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"). | ||
* You may not use this file except in compliance with the License. | ||
* A copy of the License is located at | ||
* | ||
* http://aws.amazon.com/apache2.0 | ||
* | ||
* or in the "license" file accompanying this file. This file is distributed | ||
* on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either | ||
* express or implied. See the License for the specific language governing | ||
* permissions and limitations under the License. | ||
*/ | ||
|
||
package software.amazon.awssdk.core.async; | ||
|
||
import software.amazon.awssdk.annotations.SdkPublicApi; | ||
import software.amazon.awssdk.utils.SdkAutoCloseable; | ||
|
||
/** | ||
* An extension of {@link AsyncRequestBody} that is closable. | ||
*/ | ||
@SdkPublicApi | ||
public interface CloseableAsyncRequestBody extends AsyncRequestBody, SdkAutoCloseable { | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.