Add Qwen3.5 Quark GPTQ export support for hybrid attention models

[Vishal Jain (VishalX)]

Summary

Enables end-to-end ONNX export of Qwen3.5 models when the source weights are pre-quantized with AMD Quark (GPTQ + pack_method=reorder). Qwen3.5 uses a hybrid architecture that interleaves standard full-attention layers with GatedDeltaNet linear-attention layers.

Two changes required for quantized hybrid checkpoints:

1. make_matmul() returns the actual emitted node name — for pre-quantized weights this is <basename>NBits rather than <basename> — but all linear-attention call sites discarded the return value and used the original basename string to construct downstream output references. This produced a silently disconnected ONNX graph for quantized models.
2. The quantized weight loader had no representation for linear-attention layers. Loading a hybrid Qwen3.5 checkpoint raised NotImplementedError on every linear_attn.* weight key.

Additionally, the MTP head build is now skipped when a quantization_config is present on the HuggingFace config, since pre-quantized checkpoints do not include MTP weights while mtp_num_hidden_layers in the config would otherwise cause the builder to attempt loading them.

Tested with Qwen3.5-4B and Qwen3.5-9B GPTQ INT4 checkpoints. Text-only and vision-language inference verified with latest ONNX Runtime.

Changes

src/python/py/models/builders/qwen.py

Fix make_matmul() return-value capture in linear-attention projections

make_matmul() emits a MatMulNBits node for pre-quantized weights whose name has an NBits suffix (e.g. /model/layers.0/linear_attn/qkv_proj/MatMulNBits). All five call sites in the Qwen3.5 linear-attention builder were discarding the return value, then using the original basename to form output_0 references — pointing to a non-existent node and corrupting the graph. Fixed in:

• make_linear_attention_input_proj — 4 sites: in_proj_qkv, in_proj_z, in_proj_b, in_proj_a
• make_linear_attention_output_proj — 1 site: out_proj

Skip MTP head for pre-quantized checkpoints

Qwen3_5ForConditionalGeneration configs carry mtp_num_hidden_layers: 1, but Quark/GPTQ checkpoints do not include MTP weights. The builder now checks for quantization_config on the HuggingFace config and skips the MTP head build when present, avoiding a ValueError at weight loading time.

src/python/py/models/loaders/base.py

Add QuantizedLinearAttention and wire it into the loader pipeline

• New QuantizedLinearAttention class — holds the five quantized projections of a GatedDeltaNet layer (in_proj_qkv, in_proj_z, in_proj_a, in_proj_b, out_proj as QuantizedTensorModule) plus non-quantized fields (conv1d, norm as TensorModule; plain tensors A_log, dt_bias).

• linear_attn field on QuantizedDecoderLayer — every decoder layer now holds a QuantizedLinearAttention() instance alongside the existing self_attn and mlp. It remains empty for full-attention layers and is populated for linear-attention layers.

• 20 new regex handlers in QuantizedModel.__init__ — covering qweight, scales, qzeros, g_idx for each of the five quantized projections, plus A_log, dt_bias, conv1d.weight, and norm.weight. These slot into the existing tensor-map dispatch mechanism with no changes required to QuarkModel.

• quantized_tensor_modules() and repack_quantized_tensors() updated — both iteration passes now include layer.linear_attn alongside layer.self_attn and layer.mlp, so unpack/repack and property-setting cover linear-attention weights automatically.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟡 Changes recommended

The new MTP skip condition is overly broad (based on attribute presence rather than quant_method), risking unintended disabling of MTP export for other quantized flows.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Enables ONNX export for Qwen3.5 hybrid-attention models when using pre-quantized (Quark/GPTQ) checkpoints by fixing linear-attention graph wiring and extending the quantized weight loader to recognize linear_attn.* tensors.

Changes:

• Capture and use make_matmul()’s returned node name for all Qwen3.5 linear-attention projections to prevent disconnected output_0 references for MatMulNBits nodes.
• Extend the quantized loader’s tensor-key dispatch to support linear_attn projections and associated non-quantized tensors, and include them in quantized-module iteration/repacking.
• Skip building the MTP head when quantization_config is present (intended to avoid missing-weight failures for certain pre-quantized checkpoints).

File summaries

File	Description
src/python/py/models/loaders/base.py	Adds QuantizedLinearAttention, wires linear_attn.* keys into quantized tensor dispatch, and includes linear-attention modules in quantized iteration/repack passes.
src/python/py/models/builders/qwen.py	Fixes linear-attention matmul name propagation for quantized exports and introduces logic to skip MTP head for quantized configs.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Vishal Jain (VishalX)]

kunal-vaishnavi

[kunal-vaishnavi]

/azp run macOS arm64 Build

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[kunal-vaishnavi]

Why do we need these changes to store the returned output from make_matmul?

[Vishal Jain (VishalX)]

make_matmul returns the actual node name, and for quantized weights that name differs from the basename we pass in. make_matmul_nbits names the node f"{basename}NBits", so the output tensor is .../MatMulNBits/output_0, not .../MatMul/output_0.

The old linear-attention code discarded the return value and hardcoded f"{basename}/.../MatMul/output_0". That worked for dense MatMul, but for a pre-quantized (Quark/GPTQ) export the downstream Transpose/Split/Reshape nodes referenced a tensor that did not exist, leaving a disconnected output_0. Using the returned name fixes that and matches how the rest of this file consumes make_matmul (e.g. the shared-expert gate and the MTP fc).

[Vishal Jain (VishalX)]

kunal-vaishnavi let me know if this needs a rebase