feat: Local Cloud E2E, project API keys, and index capacity/tuning ph…

[varshith-Git]

…ases (S8-S11)

Local Cloud E2E verification:

• Real Docker stack (Postgres+migrations+PostgREST+2 valori-node workers+Next.js Cloud API) proving Python SDK -> Cloud API -> project auth -> worker -> real node end-to-end.
• 6 real production bugs found and fixed, including a plaintext VALORI_AUTH_TOKEN log leak on worker startup (config.rs Debug impl now redacts auth_token/embed_api_key).

Project-scoped API keys (P1/P2.x): architecture doc, phase docs.

S8 — Deterministic Restart Integrity:

• Root-caused and fixed the state-hash-across-restart divergence: create_collection()/drop_collection() bypassed the durability layer; a secondary buffered-single-event-commit-never-flushed gap on graceful shutdown. Both fixed (commit_and_apply_ns + new flush_pending_events() wired into shutdown).

S9 — Resource Capacity & Plan Limits:

• Real Docker capacity benchmarks against the actual valori-node binary. Found max_records_per_project=1,000,000 is ~34x too high (measured safe limit ~29,300 vectors, search-latency-bound). Derived capacity model, proposed (not applied) plan limits.

S10 — IVF/BQ Capacity Validation:

• Real benchmarks found neither IVF nor BQ meaningfully beats BruteForce at Free-tier scale. RECOMMENDED (not applied): Free-tier quota stays BruteForce-derived.

S11 — Index Tuning & Product Defaults:

• Audited all 4 index implementations; found IVF/HNSW have real snapshot/restore code Engine::try_recover() never calls.
• Added BqConfig{pool_factor,min_candidates} + VALORI_BQ_POOL_FACTOR/ VALORI_BQ_MIN_CANDIDATES env vars (additive, default-preserving).
• IVF sweep: no config achieves 25% latency win; recovery scales linearly with n_list independent of n_probe.
• BQ sweep: min_candidates=10000 reaches Recall@10=0.99 (up from 0.48).
• HNSW: ties BruteForce on latency, 24-28x worse insert, ~150x worse recovery -- does not clear its own bar, 50K point not run.
• Recommendation: BruteForce remains the Free-tier default.

Verification: cargo test --workspace 1189/1189 passed, clippy clean, tsc/npm build clean. No production quotas, pricing, or provisioning changed in S8-S11.

Summary

What does this change do, and why?

Determinism Impact

• No determinism impact
• Improves determinism
• Changes arithmetic behavior (Q16.16 / FxpScalar)

If arithmetic behavior changes, explain why and what compatibility fixtures were updated.

no_std / Kernel ABI

Only applies if crates/valori-kernel or crates/valori-core were touched.

• N/A — kernel/core not touched
• No use std:: added inside valori-kernel/src/; std-only code is gated behind #[cfg(feature = "std")]
• cargo build -p valori-kernel --target wasm32-unknown-unknown passes locally

Standalone + Cluster Parity

Only applies if this adds or changes an HTTP endpoint or a KernelEvent.

• N/A — no endpoint/event change
• Handler added to both server.rs and cluster_server.rs
• Write path in cluster mode goes through raft.client_write(), not a direct engine lock
• cargo test -p valori-node --test route_parity passes (or the allowlist was updated with a reason)

Cross-Architecture Behavior

Tested on:

• x86
• ARM / Apple Silicon
• Embedded / Jetson

Tests

• cargo test -p valori-kernel -p valori-node passes
• cargo fmt --all -- --check and cargo clippy --workspace --all-targets -- -D warnings pass

Include reproduction / validation cases below.

UI (if applicable)

• N/A — no UI change
• Verified in both dark and light mode (semantic tokens only, no hardcoded colors)

Notes

If this relates to forensic or evaluator tracks, confirm that logic is kernel-appropriate.

Anything deferred to a follow-up, and which phase/issue owns it.