Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This does not add scalar quantization: the accelerated scalar codec, service registration, and benchmark selector already exist. It fixes the unsigned byte encoding supplied to the intermediate CAGRA graph build.
[0, 127], matching the unsignedCuVSMatrix.DataType.BYTEcontract.Why
The old helper emitted signed values in
[-64, 63]. Its conversion used(byte) (value & 0xff), which is a bitwise no-op after casting the result back to a Java byte. Because the cuVS byte matrix is exposed askDLUInt, negative codes were read as192..255while zero remained0, introducing a discontinuity into CAGRA distances.The direct unsigned formula is the intended centered code plus 64:
For Euclidean distance, that constant shift preserves the intended pairwise distances while satisfying the unsigned matrix contract.
Float.MIN_VALUEwas also replaced because it is the smallest positive float, not a negative lower bound.Usage
No benchmark-harness or YAML API change is required. The existing benchmark selector is:
This branch is based directly on the current NVIDIA/cuvs#2476 head,
8faae5092.Compatibility and scope
Validation
Exact branch head:
cuvs-lucenemodule: 346 tests across 33 suites, 0 failures, 0 errors, 30 skips../build.sh java lucene --run-java-testscompleted successfully, including Panama binding generation and local artifact installation.mvn -q spotless:checkandgit diff --checkpassed.fern/build_docs.sh checkvalidated all 284 MDX files with 0 errors. Its two warnings were the existing unauthenticated redirect check and light-mode accent contrast warning.Historical Euclidean single-run ablations of this same three-file logical patch on a later integration base showed Deep1B-1M recall at
efSearch=1500increase from 92.4417% to 94.9120%, while Jasper-1M moved from 97.9313% to 97.9920%. These are supporting observations, not exact-head performance claims or evidence that scalar quantization should become a default.