Support Qwen3.5 MoE INT4-QAT

[ShuZihan]

Summary

Support Qwen3.5 MoE INT4-QAT for routed experts.

This PR adds:

• Qwen3.5 fused 3D routed expert support in convert_hf_to_int4_direct.py
• fused expert split from gate_up_proj / down_proj into per-expert 2D gate_proj / up_proj / down_proj
• runtime Megatron-to-HF weight sync support for Qwen3.5 fused experts
• EP global expert id preservation during runtime weight sync
• Qwen3.5 INT4-QAT run script
• English and Chinese low-precision docs
• unit tests for offline conversion, runtime split, ignore rules, and non-fused compatibility

Scope

Current support targets routed experts only.

INT4:
- mlp.experts.{i}.gate_proj
- mlp.experts.{i}.up_proj
- mlp.experts.{i}.down_proj

BF16:
- self_attn
- linear_attn
- conv1d
- shared_expert
- mlp.gate
- visual
- mtp
- embed
- norm
- lm_head

MTP training and visual INT4-QAT are not included in this PR.

Why

Qwen3.5 MoE routed experts can be stored as fused 3D tensors in Hugging Face checkpoints:

mlp.experts.gate_up_proj
mlp.experts.down_proj

The INT4 compressed-tensors / SGLang MoE path expects per-expert 2D weights. This PR adds the missing split path so Qwen3.5 can use the same experts-only INT4-QAT flow.

Test Plan

Unit and static checks:

• python3 -m compileall -q tools/convert_hf_to_int4_direct.py slime/backends/megatron_utils/megatron_to_hf/qwen3_5.py slime/backends/megatron_utils/megatron_to_hf/processors/quantizer_compressed_tensors.py slime/backends/megatron_utils/update_weight/common.py tests/test_qwen35_int4_fused_expert_quantizer.py
• bash -n scripts/low_precision/run-qwen3.5-35B-A3B-int4-qat-rl.sh
• git diff --check refs/remotes/thudm/main...HEAD
• python -m pytest tests/test_qwen35_int4_fused_expert_quantizer.py -q
  • Result: 7 passed

Server validation:

• Convert Qwen3.5-35B-A3B HF checkpoint to Megatron BF16 torch_dist
• Convert Qwen3.5-35B-A3B HF checkpoint to experts-only INT4 checkpoint
• Verify INT4 checkpoint only packs routed experts
• Run Qwen3.5-35B-A3B INT4-QAT-RL smoke
• Confirm checkpoint is saved under Qwen3.5-35B-A3B_slime

Validated on 4 x H20 141GB. The Qwen3.5-35B-A3B INT4-QAT-RL smoke test completed successfully and saved a checkpoint under Qwen3.5-35B-A3B_slime.

Smoke config: num-rollout=2, rollout-batch-size=2, n-samples-per-prompt=2, rollout-max-response-len=512.

Server reproduction:

export BASE_FOLDER=/mnt/slime-qwen35
export HF_MODEL=/mnt/public_data/Qwen/Qwen3.5-35B-A3B
export MASTER_ADDR=127.0.0.1

source scripts/models/qwen3.5-35B-A3B.sh

PYTHONPATH=/root/Megatron-LM python tools/convert_hf_to_torch_dist.py \
  "${MODEL_ARGS[@]}" \
  --hf-checkpoint "$HF_MODEL" \
  --save "$BASE_FOLDER/Qwen3.5-35B-A3B_torch_dist"

python tools/convert_hf_to_int4_direct.py \
  --model-dir "$HF_MODEL" \
  --save-dir "$BASE_FOLDER/Qwen3.5-35B-A3B-INT4" \
  --group-size 128 \
  --is-symmetric \
  --max-workers 1

bash scripts/low_precision/run-qwen3.5-35B-A3B-int4-qat-rl.sh

Follow-up work:

• Accuracy validation against BF16 rollout on math eval sets.
• Longer INT4-QAT-RL runs to check reward/loss/KL stability.
• Resume-from-checkpoint validation.
• Multi-node and non-colocate rollout/training validation.
• Scaling validation for Qwen3.5-122B-A10B and Qwen3.5-397B-A17B.
• Throughput and memory comparison between BF16 rollout and INT4 rollout.
• Future support for MTP INT4-QAT and visual INT4-QAT, if needed.