Skip to content

[Archive] Preserve original PR-2653 graph processing head - #4

Draft
nvzm123 wants to merge 2 commits into
mainfrom
archive/pr2653-before-pr1-sync-20260922
Draft

nvzm123 wants to merge 2 commits into
mainfrom
archive/pr2653-before-pr1-sync-20260922

Conversation

@nvzm123

@nvzm123 nvzm123 commented Sep 23, 2026

Copy link
Copy Markdown
Owner

Purpose

This fork-only draft preserves the original NVIDIA/cuvs#2653 head at f92d07a7d, before that work was synchronized with PR 2476 and reworked into nvzm123/cuvs#1.

The archived branch is unchanged. This PR exists for historical inspection and comparison; it is not intended to merge.

Preserved work

  • Bounded parallel materialization of CAGRA adjacency rows into Lucene graph objects.
  • A temporary host copy for concurrent reads of device-backed adjacency, capped by a 4 GiB logical payload budget.
  • Bounded parallel level-0 graph serialization with ordered output and serial higher layers.
  • Worker lifecycle, interruption, and failure propagation through BoundedParallelExecutor.
  • Serial/parallel byte-equivalence, resource-failure, executor, and persisted-index coverage.
  • Documentation for the writerThreads behavior.

The exact archived delta is two commits and 19 files (+1520/-120) relative to its original base e0f8a4eb4.

Hardening scope

This snapshot hardens resources introduced by its graph-postprocessing path: a failed device-to-host copy closes the temporary host allocation and preserves a distinct close failure; successful staging copies and executors use scoped cleanup; serialization memory is bounded; and focused tests cover those behaviors.

It does not address the broader dataset-to-CagraIndex ownership issue later identified in PR 2476. The unquantized, binary, scalar, subset, single-vector, and upper-layer paths still retain their older tail-only cleanup behavior. A CAGRA build failure can therefore leave the input dataset unclosed, and a later graph/write failure can leave the index and its owned dataset unclosed.

Relationship to current work

This snapshot predates and does not contain the current PR-2476 implementation. Fork PR #1 is the later synchronized version of the bounded graph materialization/serialization work. This archival PR is the appropriate place to inspect what the earlier PR-2653 head itself contained.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant