Repository navigation
Release 3.0.0 with corrected anchor-batched precision - #774
Merged
Merged
Conversation
Float32 working copies lost the original dtype, so low-precision rows got the matrix-product cdist kernel and float32's normalization eps, changing losses and gradients from the full loss. Chunks now stay in float32 and tell those two sites the input dtype. Verified: reproduction matches, four mutations caught, check and changed tests pass.
Set the version, citation, and lock metadata, add consolidated release notes, and point installation commands and source links at 3.0.0. Replace unmeasured memory claims with the GPU benchmark results.
A dtype context covered the whole chunk, so float32 references took a float16 query's normalization eps, changing the loss and adding reference gradients. Widened tensors now record their source dtype in a Tensor subclass, which follows them through slicing, indexing, and normalization and replaces the context. Verified: reproductions match, seven mutations caught, CPU tests and checks pass. CUDA evidence is pending.
Under ambient autocast, float16 loss terms overflowed the running total to inf, and backward formulas narrowed float32 gradients. Forward and backward passes now run without autocast, so chunks stay in float32. Verified: reproduction matches, ten mutations caught, CPU tests and checks pass. CUDA evidence is pending.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release stable
3.0.0with corrected anchor-batched precision semantics. The earlier chunk implementation could change low-precision normalization and Euclidean-distance kernel choices, mis-normalize mixed-dtype reference tensors, and overflow its running loss under autocast.Each widened tensor now retains its own source-dtype behavior. Anchor batching disables autocast for its entire computation and backward, and restores plain tensors at its public return and gradient hooks. Tests cover nearby/tiny rows, mixed query/reference dtypes, swap triplets, learned parameters, and ambient float16/bfloat16 autocast. The public contract documents the resulting autocast dtype behavior and the measured runtime/memory costs.
The stable promotion also updates package/lock metadata, citation metadata, installation instructions, tagged documentation links, and consolidated release notes. Dependency versions are unchanged.
Validation:
9793227: 1,229 tests passed, zero failures.9793227: 1,231 tests and 2,471 subtests per run; zero failures, errors, or skips, with four device-specific deselections.NCCL_P2P_DISABLE=1; one used default NCCL transport. The GPUs share a NUMA node; this does not establish default-transport behavior on the earlier SYS-topology hosts.af38dc9adds only seven design-document lines after the tested source commit. Source, tests, dependencies, packaging and workflows are unchanged.Low-precision anchor batching retains its documented tradeoff: at 2,048 rows and 128 dimensions, float16 triplet peak memory is about 55 MiB versus 2,118 MiB for full backward; float16 NT-Xent batching costs about one-third more time after preserving dtype behavior. The optimization remains optional.