Repository navigation
Clarify CRT minimumPartSizeInBytes Javadoc for downloads and default sizes - #7484
Merged
Merged
Conversation
minimumPartSizeInBytes also sets the part size of ranged GET downloads, not only multipart uploads. Since 2.39.6 the SDK no longer passes a default part size to the CRT, so downloads use a part size the CRT chooses. Update the default descriptions of minimumPartSizeInBytes and initialReadBufferSizeInBytes, and rename the uploadPartSize parameter. Fixes #5158
adev-code
added
the
no-api-surface-area-change
Fred1155
reviewed
Oct 26, 2026
Review feedback on #7484: copyObject on the CRT-based client also uses this part size for multipart copy (CopyObjectHelper, default 8MB). Docs-only changes don't need a changelog entry.
adev-code
added
the
changelog-not-required
Fred1155
approved these changes
Oct 27, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to no response for status checks
Oct 27, 2026
|
This pull request has been closed and the conversation has been locked. Comments on closed PRs are hard for our team to see. If you need more assistance, please open a new issue that references this one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Fixes #5158
Docs changed by this PR:
S3CrtAsyncClientBuilder.minimumPartSizeInBytesS3CrtAsyncClientBuilder.initialReadBufferSizeInBytesS3CrtAsyncClientBuilder.javaIn #5158, a customer streamed large S3 objects to slow clients with
AsyncResponseTransformer.toPublisher()on the CRT-based S3 client, and direct memory filled up with the file's data. Setting a smallerminimumPartSizeInBytesfixed it, because this setting also sets the part size for downloads (ranged GET requests), not only for uploads. The Javadoc doesn't say that: it only talks about upload limits and names the parameteruploadPartSize. The team agreed the documentation should clarify this, and the last comment on the issue confirms the parameter name is the remaining gap.The Javadoc also says "By default, it is 8MB", which is no longer true for downloads. Since #6588 (first released in 2.39.6), the SDK only passes a part size to the CRT when one is set. Otherwise the CRT picks the download part size from the memory limit and target throughput (calculation in the aws-c-s3 version bundled with aws-crt-java 0.49.0).
Modifications
In
S3CrtAsyncClientBuilder:minimumPartSizeInBytes(Long): says the part size applies to uploads, copies and downloads (copyObjecton the CRT-based client uses it for multipart copy, seeDefaultS3CrtAsyncClientandCopyObjectHelper). Default: 8MB for uploads and copies, and for downloads a size the CRT chooses based onmaxNativeMemoryLimitInBytesandtargetThroughputInGbps. The S3 limits link now says it's guidance on part sizes. Renamed the parameteruploadPartSizetopartSizeInBytes(parameter names are not part of the binary or source API, so this doesn't break callers).initialReadBufferSizeInBytes(Long): default is 10 times theminimumPartSizeInBytesvalue, or 80MB if no part size is set, which matchesS3NativeClientConfiguration.Testing
mvn install -pl :s3with tests passes (1870 JUnit + 108 TestNG tests, 0 failures, 0 Checkstyle violations, japicmp passes; SpotBugs was skipped in this local build). Dependencies were built with-DskipTests, and the s3 tests ran with dummy AWS credentials in the environment because some tests need them.javadoc -Xdoclint:allonS3CrtAsyncClientBuilder.java: no errors, and both new{@link}references resolve.License