Throughput-aware scheduling + ring-plan service + signed model registry & generic ModelRuntime adapter

[onlyzhynx]

Rebased onto current master (the ModelRuntime / one-engine-every-model commit). origin/master is an ancestor — conflict-free; the only file overlap is phase0/confidence.py, byte-identical on both sides.

Heads-up on scope + honesty: this branch grew past its original title. It now carries two things: (1) the in-house moat work (throughput scheduling, ring-plan service, receipt export, confidence gate) and (2) a model-onboarding layer (signed registry + a GenericHFRuntime impl of your ModelRuntime + an add-model tool). The runtime piece is interface-complete and unit-green but NOT GPU-validated — I did not run your step-2 spike. Happy to split this into two PRs, hold the runtime half for VllmRuntime first, or have you take the runtime adapters — your call. Flagging before you invest review time.

Moat layer (in-house; orthogonal to the ModelRuntime sprint)

• Throughput-aware ring selection (shard/throughput.py, shard/perf_store.py): picks the ring maximizing estimated tok/s, not the first feasible one (WAN + per-layer compute + draft + accept; exact subset enumeration for small rings). PerfStore learns fleet behavior via EWMA, cold-starts on 3090/4090 priors; atomic writes; largest-remainder conserves exactly. Default plan stays latency-only for byte-compat; objective:"tok_s" opts in.
• Ring-plan HTTP service (phase0/scheduler_svc.py): the bridge c0mpute calls for a plan (subset + order + per-stage blocks) from live VRAM+RTT, plus /telemetry to learn. Stdlib only.
• Auto-planned demo ring (phase0/plan_ring.py): plans the libp2p ring from live VRAM+RTT.
• Verified per-stage receipt export (phase0/specpipe.py --receipts-out): writes verified receipts as JSON for the c0mpute payout path. Only fires under --pipe + --receipts.
• Confidence-scheduled depth (phase0/specpipe.py, confidence.py): adapts pipeline depth from the acceptance EMA so a bad draft streak doesn't waste WAN traversals. K stays fixed (CUDA-graph shape K+1); only depth changes → lossless (output bit-identical). Result dict gains mean_conf_depth / final_confidence for A/B.
• Layer-count correction (plan_ring.py, launch_swarm.py): gpt-oss reconciled to its real 36 layers; NLAYERS no longer a GLM-78 holdover applied globally.
• GLM-path drafting (research, behind flags): MLA-latent KV merge + an MTP drafter scaffold. MTP is GLM-5.2-specific (NEXTN head), behind a flag, n-gram default; K>1 rollout is scaffold-only, not validated past k=1.

Model-onboarding layer (needs your ack — see heads-up)

• Single signed model registry — shard-models/1 (shard/registry.py, phase0/publish_registry.py, registry/models.json): a model defined once, both repos read it; layer count / bytes-per-layer / quant / engine path / adapter / tokenizer stop living in three places that drift (the gpt-oss 120-vs-36 bug). Signed with the publisher Ed25519 key (reuses manifest.py's canonical+sign path); consumers pin the pubkey and fail closed on bad sig, pinned-pubkey mismatch, unknown adapter, duplicate id, non-positive layerCount. plan_ring.py resolves from the registry; the old MODEL_LAYERS dict is deleted.
• Generic HF ModelRuntime adapter (phase0/hf_runtime.py): GenericHFRuntime implements your shard/node.py over the proven pipeline.load_stage/run_block — the universal HF fallback, any arch transformers can load (incl. gpt-oss) with no new engine code. make_runtime(adapter, …) maps the registry adapter field to an impl; generic-vllm falls back here until VllmRuntime lands. One adapter behind the firewall — coordinate_pipe rides on it unchanged. Not a competing interface; implements yours.
• add-model tooling + runbook (phase0/add_model.py, docs/runbooks/add-a-model.md): config → fit math (gbPerLayer/kvGbPerLayer, dense and MoE) → emit+merge+sign the registry row (version bumped). Prints the operator steps it can't do — quantize+host+CID, tokenizer round-trip, the 3-stage smoke.

Tests (all offline, no GPU)

scheduler_svc (6), plan_ring (6), throughput (29), registry (11, incl. a post-sign layerCount tamper rejected by the signature — the gpt-oss-drift guard), hf_runtime (22, conformance via a torch-free fake), add_model (16, incl. a new MoE model → signed verifiable row with zero engine code), cross-repo agreement (29, drives both repos + asserts identical KNOWN_ADAPTERS), plus confidence.py / mtp_draft.py self-tests.

What "green" does NOT cover (stated plainly)

• No GPU validation of GenericHFRuntime — a sliced layer block running on hardware and matching a full-model reference is your step-2 spike, not done here. This PR is the torch-only reference for it.
• MLA-latent / MTP not proven on-box; quantize + 3-stage smoke in the add-model flow are the irreducible/metered remainder (only the data-emission half is tested).

Out of scope (follow-up)

• VllmRuntime behind ModelRuntime + the GPU spike that gates it (MODEL_RUNTIME.md 2-3).
• Deriving weight keys from the manifest weight_map in shard/fetch.py (step 4) — flagged, not grabbed.
• Batch-aware plan_ring sizing now that the engine serves B streams.

[joelovestech]

Hey — went through the prod-improvements branch, solid work. The throughput-aware ring selection is the right call and it's model-agnostic — the order-independence insight (reuse the latency loop, let compute drive selection) is clean, and your 3090/4090 priors match the fleet you're targeting, so cold-start should be decent until the perf store learns real numbers.

One thing on the drafting side, for the M2.5 target: the MTP drafter you scaffolded is GLM-5.2-specific — it leans on GLM's native NEXTN head (layer 78), which only exists because 5.2 was trained with it. You can't add an MTP head to a model that wasn't trained with one, so it won't port to MiniMax M2.5 unless M2.5 actually exposes a NEXTN/num_mtp head — which your m25_config_probe was checking for (num_nextn_predict_layers / num_mtp_modules). If that came back empty, M2.5 has no MTP head and the draft path stays n-gram, which is what launch_ngram already does. So for the M2.5 target the real draft lever is n-gram / prompt-lookup acceptance, not the MTP rollout.

If you're keeping a parallel GLM-5.2 path (since you pulled the ring_mtp work), one heads-up: you nailed the single-step convention (post-model.norm hidden, [enorm(emb); hnorm(h)] — the one that costs 0.86→0.51 on pre-norm). But the K>1 autoregressive rollout invents two conventions that aren't validated — it re-applies model.norm to the MTP block's own output to chain steps (model.norm is the main trunk's final norm, not part of the MTP recurrence), and it runs the block at start_pos 0,1,2… instead of the true sequence offsets (wrong RoPE on an attention layer). Neither breaks correctness — the ring verifies every token — but both silently tank acceptance, the whole reason to run MTP. ring_mtp only validated k=1; worth sweeping the K>1 conventions before trusting a multi-token number.

Smaller and model-agnostic: acceptance isn't really a per-model scalar — it climbs with context — so the tok/s estimate drifts long. Non-blocking. Nice branch overall.