Skip to content

[AMD] Qwen3.5-FP8 MI355X SGLang disagg perf tuning: Docker image update / Qwen3.5-FP8 MI355X SGLang disagg 性能调优 更新Docker image #2094

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ChangLiu0709 wants to merge 2 commits into
base: main
Choose a base branch
from
+18 −3
Open

[AMD] Qwen3.5-FP8 MI355X SGLang disagg perf tuning: Docker image update / Qwen3.5-FP8 MI355X SGLang disagg 性能调优 更新Docker image #2094

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
db4487a
[AMD] Qwen3.5-FP8 MI355X SGLang disagg: image bump, node fix, runner …
Jul 6, 2026
f557d68
Remove unintended files
Jul 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions configs/amd-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -350,7 +350,7 @@ qwen3.5-fp8-mi355x-atom-mtp:
- { tp: 8, ep: 1, conc-start: 4, conc-end: 256, spec-decoding: mtp }

qwen3.5-fp8-mi355x-sglang-disagg:
image: lmsysorg/sglang-rocm:v0.5.11-rocm700-mi35x-20260511
image: lmsysorg/sglang:v0.5.14-rocm720-mi35x
model: Qwen/Qwen3.5-397B-A17B-FP8
model-prefix: qwen3.5
runner: mi355x-disagg
Expand All @@ -365,7 +365,7 @@ qwen3.5-fp8-mi355x-sglang-disagg:
search-space:
# Matches qwen3.5-fp8-mi355x-sglang TP8/EP1 low-concurrency sweep
- spec-decoding: "none"
conc-list: [ 8, 16, 32, 64, 128, 256, 512 ]
conc-list: [ 8, 16, 32, 64, 128 ]
prefill:
num-worker: 1
tp: 8
Expand Down Expand Up @@ -396,7 +396,7 @@ qwen3.5-fp8-mi355x-sglang-disagg:
# MoRI is added to is_deepep_class_backend() or shared-slot
# accounting is reconciled.
- spec-decoding: "none"
conc-list: [ 8, 16, 32, 64, 128, 256, 512 ]
conc-list: [ 8, 16, 32, 64, 128 ]
prefill:
num-worker: 1
tp: 8
Expand Down
6 changes: 6 additions & 0 deletions configs/runners.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,12 @@ labels:
- mi355x-amds_06
- mi355x-amds_07
- mi355x-amds_08
- mi355x-amds_12
- mi355x-amds_14
- mi355x-amds_16
- mi355x-amds_17
- mi355x-amds_18
- mi355x-amds_19
gb200:
- gb200-nv_0
- gb200-nv_1
Expand Down
9 changes: 9 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4556,3 +4556,12 @@
description:
- "Add high concurrency configs"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1994

- config-keys:
- qwen3.5-fp8-mi355x-sglang-disagg
description:
- "Bump Qwen3.5-FP8 MI355X SGLang disagg image from lmsysorg/sglang-rocm:v0.5.11-rocm700-mi35x-20260511 to lmsysorg/sglang:v0.5.14-rocm720-mi35x (ROCm 7.0 → 7.2, sglang v0.5.11 → v0.5.14)."
- "Fix disagg SLURM exclude list: add mia1-p01-g09 (broken pyxis) to submit.sh — was missing from multi-node path, causing prefill worker 503 failures."
- "Expand mi355x-disagg runner pool from 3 to 9 nodes."
- "Concurrency sweep: 1K1K and 8K1K both [ 8, 16, 32, 64, 128 ]."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2092

@claude claude Bot Jul 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 WARNING: pr-link points to #2092, which is the CLOSED pre-rebase predecessor of this PR — the changes actually land via this PR (#2094).
Why it matters: The perf changelog is the traceability record for config changes; linking to a closed, unmerged PR breaks that trail.
Fix:

Suggested change
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2092
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2094

@claude claude Bot Jul 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The new perf-changelog entry sets pr-link: .../pull/2092, but this PR is #2094 — the entry's bullets (image bump v0.5.11→v0.5.14, mia1-p01-g09 exclude, mi355x-disagg pool expansion, conc-list trim) exactly describe this PR's diff. Neighboring entries follow the convention of linking to their authoring PR (e.g. line 4558 → #1994). Bump the pr-link to #2094 so future perf-regression traces land on the right review context.

Extended reasoning...

What the bug is. The new entry in perf-changelog.yaml (line 4567) declares pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2092, but this PR is #2094. The bullets inside the same entry describe exactly this PR's four-part diff: (a) image bump lmsysorg/sglang-rocm:v0.5.11-rocm700-mi35x-20260511 → lmsysorg/sglang:v0.5.14-rocm720-mi35x (matches configs/amd-master.yaml), (b) mia1-p01-g09 added to SLURM_EXCLUDE_NODES in benchmarks/multi_node/amd_utils/submit.sh, (c) mi355x-disagg runner pool grown from 3 to 9 nodes in configs/runners.yaml, and (d) 1K1K/8K1K conc-list trimmed to [8, 16, 32, 64, 128]. So the entry is unambiguously documenting this PR, not a predecessor.\n\nWhy it's #2094 not #2092. The PR metadata title is verbatim [AMD] Qwen3.5-FP8 MI355X SGLang disagg perf tuning: Docker image update / ..., which matches HEAD commit db4487a ([AMD] Qwen3.5-FP8 MI355X SGLang disagg: image bump, node fix, runner pool, concurrency sweep). Recent git log shows the surrounding merged PRs — #2006, #2007, #1994, etc. — but no #2092. The neighboring changelog entries follow a strict convention: line 4558 → #1994 (matches commit b7ffc28 Update Minimax M3 B300 FP4 vllm (#1994)), line 4552 → #2006, line 4545 → #2007, and line 4538 → #2010. Every one references its own authoring PR, so #2092 for this entry is a copy-paste/typo — most plausibly a 4↔2 digit swap.\n\nImpact. No runtime behavior is affected — this is metadata used by the perf-changelog tooling to link a regression back to review context. Per the changelog's own convention (and the AGENTS.md pr-link contract), when someone traces a future Qwen3.5-FP8 MI355X SGLang disagg regression back to this entry, the hyperlink is what gives them the code-review + discussion history. Following the link to a non-existent / unrelated #2092 breaks that traceability. The description bullets themselves are detailed enough that a reader still has full context, which is why this is a nit, not a blocker.\n\nStep-by-step proof.\n1. Open perf-changelog.yaml line 4567: pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2092.\n2. Read the entry's bullets (lines 4560–4566): image bump, mia1-p01-g09 exclude, runner pool expansion, conc-list trim.\n3. Diff each bullet against the PR:\n - configs/amd-master.yaml line 353: image was v0.5.11-rocm700, now v0.5.14-rocm720 ✔\n - benchmarks/multi_node/amd_utils/submit.sh line 159: adds mia1-p01-g09 to SLURM_EXCLUDE_NODES ✔\n - configs/runners.yaml line 126: adds mi355x-amds_{12,14,16,17,18,19} to mi355x-disagg (3 → 9 nodes) ✔\n - configs/amd-master.yaml lines 368 & 399: conc-list shrunk to [8, 16, 32, 64, 128] ✔\n4. Read PR metadata: number = 2094, title matches db4487a.\n5. Grep neighboring changelog entries for the pr-link convention: lines 4538/4545/4552/4558 point to #2010/#2007/#2006/#1994 — each matches its authoring commit's (#XXXX) suffix. Conclusion: the pr-link is supposed to be the authoring PR, and #2094 is that PR.\n\nFix. Change pull/2092 → pull/2094 on perf-changelog.yaml:4567. One-character edit; no runtime or CI impact.

Loading