Skip to content

[DRAFT] Stream based upload retry support #6368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

[DRAFT] Stream based upload retry support #6368

wants to merge 2 commits into from

Conversation

zoewangg
Copy link
Contributor

@zoewangg zoewangg commented Aug 22, 2025 •
edited
Loading

Motivation and Context

Currently retry is not supported for putObject operation in S3 multipart client if a non-file based AsyncRequestBody is configured. When a retry is attempted, the following error will be thrown. See
#6198

 A retry was attempted, but AsyncRequestBody.split does not support retries.

This pull request adds support retries for individual parts of a multipart upload in S3 multipart client through a new API: BufferedSplittableAsyncRequestBody

Code Example:

AsyncRequestBody originalBody = AsyncRequestBody.fromString("Hello World");

BufferedSplittableAsyncRequestBody retryableBody =
    BufferedSplittableAsyncRequestBody.create(originalBody);

s3Client.putObject(r -> r.bucket(TEST_BUCKET).key(TEST_KEY), retryableBody).join();

Modifications

The changes have been reviewed previously in #6313 and #6346

zoewangg and others added 2 commits August 11, 2025 11:22
@zoewangg
Stream retry support part 1: Add BufferingAsyncRequestBody that buffe…
40cedd3 
…rs the entire content and sup… (#6313)

* Add BufferingAsyncRequestBody that buffers the entire content and supports multiple concurrent subscribers

* Apply suggestions from code review


* Update existing byteBuffersAsyncRequestBody to support multicast
@zoewangg
Stream retry support part 2: Introduce a new split method in AsyncReq…
c82d494 
…uestBody that returns an SdkP… (#6346)

* Introduce a new split method in AsyncRequestBody that returns an SdkPublisher of ClosableAsyncRequestBody and use it in s3 multipart client

* Fix build and add more tests

* Refactoring based on API surface are review

* Add more tests and address comments

* Deprecate split, update documentation and add changelog entries
@zoewangg zoewangg added the api-surface-area-approved-by-team Indicate API surface area introduced by this PR has been approved by team label Aug 22, 2025
@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
76.4% Coverage on New Code (required ≥ 80%)
7.9% Duplication on New Code (required ≤ 3%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
api-surface-area-approved-by-team Indicate API surface area introduced by this PR has been approved by team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@zoewangg