Fix: stabilize depth-two chip run lifecycle

[high-cloud]

Summary

This draft preserves the complete historical runtime fix that made the DSV4 MTP-1 serving step stable in the original checkout. It is intentionally published as a behavior reference for Simpler experts, rather than presented as a conflict-free patch for current main.

The change keeps preparation, native execution, completion, and stream retirement owned by resident workers, and removes shared stderr contention from timing spans.

Historical base and current-main status

• Base commit: 3165cc89b6ea6b58a0bc01cbec2d5f72f2029c35
• Current origin/main at publication time: 66ba5c4a1100de402db103bd140a9cb106315eef
• The historical base is 217 commits behind current main.
• Conflicts with current main are expected. Please treat this draft as a migration aid and a source of lifecycle invariants, not as merge-ready code.

What changed

• Prestart the depth-two preparation and completion pools.
• Reuse one native executor pool per device context.
• Serialize stream retirement and replenishment across pipeline slots.
• Buffer host timing spans per process instead of synchronously writing every span to shared stderr.
• Add coverage for concurrent preparation, completion, shutdown, and stream-slot reuse.

Verification

Fresh host-side verification on the historical branch:

• tests/ut/py/test_worker/test_host_worker.py: 217 passed
• C++ non-hardware suite: 79/79 passed
• Focused test_run_stream_slots and test_native_run_launch_signal: 2/2 passed
• Exact-file pre-commit: headers, English-only, EOF/whitespace, clang-format, clang-tidy, cpplint, Ruff, and Pyright all passed
• git diff --check: passed

Historical real-NPU acceptance, collected from the same implementation before publication:

• DSV4 Serving MTP-1, GBS32, DP8/EP8
• 32 requests x 256 tokens = 8192 output tokens
• 137 decode rounds
• runner-end cross-rank skew p95/max: 0.224/0.953 ms
• step-end cross-rank skew p95/max: 0.340/0.982 ms
• runner-to-validate gap p95/max: 0.078/0.162 ms
• validate duration p95/max: 0.192/0.896 ms
• No event exceeded its median by 5 ms in these four categories

The NPU result above is historical evidence, not a fresh run against current main.

Related work

• Serving reproduction and analysis: [Bug] DeepSeek V4 decode step latency is unstable across ranks pypto-serving#179
• Narrower full-rank activation attempt: Fix: align full-rank chip run activation #1967
• Buffered host timing spans: Fix: redirect host spans to per-process buffered files #1945

Review request

Please help identify which lifecycle invariants still apply to current main, and whether they should be ported through the current stream/worker abstractions as smaller changes.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}