Repository navigation
Conversation
Replace the byte-at-a-time Ionic.Zlib CRC32 loop with Crc32Ieee: PCLMULQDQ folding on x86/x64 and ARM64 CRC32 instructions on .NET 8+, slicing-by-16 tables on netstandard2.0/netcoreapp3.1/net472. Covers Crc32Managed (flexible checksums), CrcCalculatorStream (x-amz-crc32, event streams). Output is bit-identical to the previous implementation.
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Replace the byte-at-a-time Ionic.Zlib CRC32 loop with Crc32Ieee: PCLMULQDQ
folding on x86/x64 and ARM64 CRC32 instructions on .NET 8+, slicing-by-16
tables on netstandard2.0/netcoreapp3.1/net472. Covers Crc32Managed (flexible
checksums), CrcCalculatorStream (x-amz-crc32, event streams). Output is
bit-identical to the previous implementation.
PCLMULQDQ folding on x86/x64 and ARM64 CRC32 instructions on .NET 8+; slicing-by-16 tables on netstandard2.0/netcoreapp3.1/net472. The vectorized code is adapted from dotnet/runtime's
System.IO.Hashing(MIT, underThirdParty/RuntimeBackports).Motivation and Context
The byte-at-a-time CRC32 dominated checksum cost on large uploads and
x-amz-crc32validation.The gain is concentrated in the large-payload cases.
Measurement: Numbers come from the serde ops/CPU-sec runner (
sdk/test/Performance/SerdeBenchmarks,--cpu-time, stub transport), geomean of 71 cases, 3 interleaved runs. Instances: m7i.metal-24xl (x86) and m7g.metal (ARM64). Settings: net8.0, 20,000-iteration warmup, measured ondevelopment8956121, which is the same code as this branch for these files. Each change is measured alone against that base. They will be re-measured at the harness's current default (100,000 warmup, net10.0, stub and https) before the PRs leave draft.Testing
Crc32IeeeTests(20): known vectors; slicing-by-16, PCLMULQDQ and ARM64 paths against a reference implementation over all lengths and offsets; incremental chunking vs one-shot;Crc32Managed, the checksum factory andCrcCalculatorStreamagainst the reference.Breaking Changes Assessment
None. Output is bit-identical.
Dry-runs
Combined dry-run of this series (
jonatnun/perf-all-v3@ 2a8cec6: #4557–#4563 together).438fa1b0-130d-4670-b74e-2df6c79bb7a1c0049dbb-e11a-4a90-9fcd-087f2dc26e93Checklist
License