Skip to content

Fix autoencoder CUDA kernel: initialise example-buffer padding bits#86

Open
drmckay wants to merge 1 commit into
cair:mainfrom
drmckay:fix/autoencoder-init-padding
Open

Fix autoencoder CUDA kernel: initialise example-buffer padding bits#86
drmckay wants to merge 1 commit into
cair:mainfrom
drmckay:fix/autoencoder-init-padding

Conversation

@drmckay

@drmckay drmckay commented Jul 1, 2026

Copy link
Copy Markdown

Fix 2 — CUDA autoencoder kernel leaves padding bits uninitialised

The bug

produce_autoencoder_example (CUDA, cuda/tools.cu) builds the example literal-vector X, a
bit-packed array of number_of_literals = 2*number_of_features bits stored in
number_of_literal_chunks = (number_of_literals-1)/32 + 1 32-bit words. When number_of_literals
is not a multiple of 32, the final word has padding bits [number_of_literals, chunks*32).

The kernel computed number_of_literal_chunks but never used it. It initialised X by clearing
the positive-literal bits one at a time and setting the negated-literal bits — touching only
[0, number_of_literals). The padding bits were never written, so they kept whatever garbage
was in the freshly mem_alloc-ed device buffer (prepare_X_autoencoder allocates X_gpu but never
copies a zeroed host array to it).

The reference: the CPU path already does it right

tmu_produce_autoencoder_example (lib/src/Tools.c):

unsigned int number_of_literal_chunks = (number_of_literals-1)/32 + 1;
// Initialize example vector X
memset(X, 0, number_of_literal_chunks * sizeof(unsigned int));   // zero the WHOLE buffer
for (int k = number_of_features; k < number_of_literals; ++k) ... // then set negated literals

The CUDA kernel was ported from this but dropped the memset (and replaced it with a positive-bit
clear loop). So GPU and CPU diverged, and only the GPU leaves padding uninitialised.

The fix

Match the CPU path: zero all number_of_literal_chunks words first, then set the negated-literal
bits. Minimal diff in cuda/tools.cu, and it also silences the
number_of_literal_chunks declared but never referenced nvcc warning.

for (unsigned int c = 0; c < number_of_literal_chunks; ++c) {
    X[c] = 0;
}
for (int k = number_of_features; k < number_of_literals; ++k) { ... }   // unchanged

Proof

Correctness is by parity: the CPU reference already zeroes the full buffer; the fix makes the GPU do
the same. Meaningful bits [0, number_of_literals) are set identically (they were cleared/zeroed
either way); only the padding bits change (garbage → 0).

  • bench/check_padding.py (branch test/gpu-ae-padding, RTX 3090): 80 literals, 3 chunks, padding
    bits 80..95 → padding bits set = 0 for all 4 targets → PASS — padding deterministically 0.
  • example/toy_autoencoder.py --compare: CPU precision 1.000 / recall 0.358, GPU precision 1.000 /
    recall 0.398 → no regression (meaningful bits unchanged; only padding went garbage → 0).
  • nvcc no longer warns number_of_literal_chunks declared but never referenced (now used).

The kernel computed number_of_literal_chunks but only set the literal bits, leaving the
final-chunk padding uninitialised. Zero the whole buffer first, matching the CPU memset.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant