fix(storage): fail the download instead of committing bad chunk data

[define-null]

Problem

A chunk committed when every file's HTTP status was not 4xx/5xx and the body copied without an I/O error. Nothing looked at the bytes. Two ways bad data was being committed and served until an assignment stopped naming it:

• A redirect. The client follows no redirects (Make HTTP requests nice #41) and error_for_status passes 3xx, so the redirect response's body was written as the table.
• A body that isn't the file — truncated, empty, or an error page under a 200. Parquet keeps its metadata in the footer, so these fail every query over the chunk with reading parquet metadata … (server_error), permanently: recovery adopts chunks by directory name and reconciliation only queues absent chunks, so a bad chunk is never re-fetched.

Fix

In download_one:

• Reject any non-2xx.
• Parse each downloaded .parquet footer with ParquetMetaDataReader — the same read the query engine does when it opens a chunk. Footer only, page data isn't decoded. parquet was already in the graph via sqd-query and sqd-query-engine; Cargo.lock is unchanged.

A rejected file fails its chunk: the staging directory is swept and the existing retry/give-up path applies unchanged.

Tests

tests/download_integrity.rs drives the real ChunkDownloader against exact raw responses. The first commit adds the tests — 4 of 7 fail on master — and the second makes them pass. The redirect case carries a valid parquet body, so only the status check can reject it; each check was confirmed load-bearing by disabling it in turn.

Not covered here

• A well-formed but wrong file: the assignment carries no size or digest (GAP-5 remainder).
• A chunk missing a table (empty file list, escaped legacy file name) still commits, and the dynamic engine answers its logs queries as empty rather than erroring. Belongs at DatasetsIndex::new.
• Post-commit damage: nothing syncs before the rename (CN-4).

🤖 Generated with Claude Code