Skip to content

fix(llm-client): disable reasoning per-model via baseURL-aware table#949

Open
DerekEXS wants to merge 1 commit into
CortexReach:masterfrom
DerekEXS:fix/disable-reasoning-multi-model
Open

fix(llm-client): disable reasoning per-model via baseURL-aware table#949
DerekEXS wants to merge 1 commit into
CortexReach:masterfrom
DerekEXS:fix/disable-reasoning-multi-model

Conversation

@DerekEXS

Copy link
Copy Markdown

Extends the single-line /qwen3|deepseek.*r1|qwq/ regex (issue #929) into a baseURL-aware rule table that injects the correct disable-thinking request parameter for each known Chinese reasoning-model family on its respective provider endpoint.

Why a table (not just a regex fix)

The same model can require different parameter shapes on different providers. For example, deepseek-v4-flash uses chat_template_kwargs.enable_thinking=false on Aliyun DashScope but thinking.type="disabled" on DeepSeek's own API. Model name alone is not enough — the table selects the right parameter per (model family, provider).

Coverage (parameters from each provider's official docs)

Family Provider Parameter
Qwen3 / QwQ DashScope / vLLM chat_template_kwargs.enable_thinking=false
DeepSeek V4 / R1 DashScope chat_template_kwargs.enable_thinking=false
DeepSeek V4 DeepSeek official thinking.type="disabled"
GLM-4.6 / GLM-Z1 Zhipu BigModel thinking.type="disabled"
Doubao thinking/seed Volcano Ark (OpenAI-compat) thinking.type="disabled"
Kimi K1.5 / K2 Moonshot thinking.type="disabled"
MiniMax M2 / abab MiniMax thinking_budget=0
Hunyuan T1 / Turbo-S Tencent enable_thinking=false
StepFun step-3 / step-2 StepFun thinking=false

API

Verification

Live test against DashScope deepseek-v4-flash with the extraction prompt, three variants:

Variant Reasoning chars Completion tokens Total
(baseline) no param 225 67 (incl. 53 reasoning) 102
chat_template_kwargs.enable_thinking=false 0 12 47
thinking.type="disabled" 0 12 47

Completion tokens -82%. DashScope deepseek-v4-flash accepts both parameters. Only this one model lane was live-verified; the rest are doc-sourced and rely on recoverJsonFromReasoning as a safety net for unverified models/providers.

Tests

test/llm-api-key-client.test.mjs — all 8 pass (6 original + 2 new: per-provider param table, deepseek-v4-flash end-to-end injection).

Fixes #929

Extends the single-line /qwen3|deepseek.*r1|qwq/ regex (issue CortexReach#929) into a
baseURL-aware rule table that injects the correct disable-thinking request
parameter for each known Chinese reasoning-model family on its respective
provider endpoint. The same model can require different parameter shapes on
different providers (e.g. deepseek-v4-flash uses chat_template_kwargs.enable_thinking
on Aliyun DashScope but thinking.type=disabled on api.deepseek.com), so a
provider-aware table is required rather than model-name matching alone.

Covers 8 families (parameters sourced from each provider's official docs):
- Qwen3 / QwQ: chat_template_kwargs.enable_thinking=false
- DeepSeek V4 / R1: provider-aware (DashScope -> chat_template_kwargs;
  DeepSeek official -> thinking.type=disabled)
- GLM-4.6 / GLM-Z1 (Zhipu): thinking.type=disabled
- Doubao thinking/seed (Volcano Ark OpenAI-compat): thinking.type=disabled
- Kimi K1.5 / K2 (Moonshot): thinking.type=disabled
- MiniMax M2 / abab: thinking_budget=0
- Hunyuan T1 / Turbo-S (Tencent): enable_thinking=false
- StepFun step-3 / step-2: thinking=false

shouldDisableReasoningForJson(model) is kept as a backward-compatible wrapper
around the new buildDisableReasoningParams(model, baseURL). recoverJsonFromReasoning
(PR CortexReach#914) remains as a safety net for models/providers not yet in the table.

Verified live on DashScope deepseek-v4-flash: reasoning content drops from
~225 chars / 53 reasoning_tokens to 0 with chat_template_kwargs.enable_thinking=false;
completion tokens -82% per extraction call.

Fixes CortexReach#929

@app3apps app3apps left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The DashScope deepseek-v4-flash fix is valuable, and the verified path looks sound. I am requesting changes because the expanded table also ships provider/model mappings that are incorrect for configurations this project supports.

  1. MiniMax M2 cannot be disabled by this rule (src/llm-client.ts:188-192). The current MiniMax OpenAI-compatible documentation explicitly says thinking cannot be disabled for M2.x, but this regex matches every MiniMax-M2* model and injects thinking_budget: 0. That cannot deliver the advertised behavior and may be ignored or rejected. Please remove/narrow the M2 and broad abab mapping unless there is a provider-specific documented endpoint and request-level verification: https://platform.minimaxi.com/docs/api-reference/text-openai-api

  2. Most entries are not actually provider-aware (src/llm-client.ts:164-207). Seven rules use a universal default, so private provider fields are sent to every OpenAI-compatible endpoint after a model-name match. There is a concrete mismatch with the built-in SiliconFlow path: SiliconFlow documents top-level enable_thinking: false for zai-org/GLM-4.6, while this table sends { thinking: { type: "disabled" } }. At best reasoning remains enabled; a strict gateway may reject the request before recoverJsonFromReasoning can help. Please positively identify provider hosts and return null for unknown endpoints, or add an explicit capability override. Add negative unknown-provider coverage and a SiliconFlow case: https://docs.siliconflow.cn/en/api-reference/chat-completions/chat-completions

  3. The Kimi matcher includes its own unsupported exception (src/llm-client.ts:179-185). The comment says kimi-k2.7-code cannot disable thinking, but /kimi.*k[12]|kimi.*thinking/i matches it and injects thinking.type=disabled. Please use a disable-capable allowlist or add an explicit exclusion plus a negative test.

A clean way forward would be to narrow this PR to the live-verified DashScope DeepSeek V4 fix, then add the other families only with provider-specific evidence and tests.

Verification performed: full npm test passed on the PR branch; npm run build passed with no generated diff; the merge result with current master also built and passed the combined LLM API-key client tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] shouldDisableReasoningForJson regex doesn’t match deepseek-v4-flash → wastes ~88-94% of output tokens on reasoning trace

2 participants