fix(llm-client): disable reasoning per-model via baseURL-aware table#949
fix(llm-client): disable reasoning per-model via baseURL-aware table#949DerekEXS wants to merge 1 commit into
Conversation
Extends the single-line /qwen3|deepseek.*r1|qwq/ regex (issue CortexReach#929) into a baseURL-aware rule table that injects the correct disable-thinking request parameter for each known Chinese reasoning-model family on its respective provider endpoint. The same model can require different parameter shapes on different providers (e.g. deepseek-v4-flash uses chat_template_kwargs.enable_thinking on Aliyun DashScope but thinking.type=disabled on api.deepseek.com), so a provider-aware table is required rather than model-name matching alone. Covers 8 families (parameters sourced from each provider's official docs): - Qwen3 / QwQ: chat_template_kwargs.enable_thinking=false - DeepSeek V4 / R1: provider-aware (DashScope -> chat_template_kwargs; DeepSeek official -> thinking.type=disabled) - GLM-4.6 / GLM-Z1 (Zhipu): thinking.type=disabled - Doubao thinking/seed (Volcano Ark OpenAI-compat): thinking.type=disabled - Kimi K1.5 / K2 (Moonshot): thinking.type=disabled - MiniMax M2 / abab: thinking_budget=0 - Hunyuan T1 / Turbo-S (Tencent): enable_thinking=false - StepFun step-3 / step-2: thinking=false shouldDisableReasoningForJson(model) is kept as a backward-compatible wrapper around the new buildDisableReasoningParams(model, baseURL). recoverJsonFromReasoning (PR CortexReach#914) remains as a safety net for models/providers not yet in the table. Verified live on DashScope deepseek-v4-flash: reasoning content drops from ~225 chars / 53 reasoning_tokens to 0 with chat_template_kwargs.enable_thinking=false; completion tokens -82% per extraction call. Fixes CortexReach#929
app3apps
left a comment
There was a problem hiding this comment.
The DashScope deepseek-v4-flash fix is valuable, and the verified path looks sound. I am requesting changes because the expanded table also ships provider/model mappings that are incorrect for configurations this project supports.
-
MiniMax M2 cannot be disabled by this rule (
src/llm-client.ts:188-192). The current MiniMax OpenAI-compatible documentation explicitly says thinking cannot be disabled for M2.x, but this regex matches everyMiniMax-M2*model and injectsthinking_budget: 0. That cannot deliver the advertised behavior and may be ignored or rejected. Please remove/narrow the M2 and broadababmapping unless there is a provider-specific documented endpoint and request-level verification: https://platform.minimaxi.com/docs/api-reference/text-openai-api -
Most entries are not actually provider-aware (
src/llm-client.ts:164-207). Seven rules use a universaldefault, so private provider fields are sent to every OpenAI-compatible endpoint after a model-name match. There is a concrete mismatch with the built-in SiliconFlow path: SiliconFlow documents top-levelenable_thinking: falseforzai-org/GLM-4.6, while this table sends{ thinking: { type: "disabled" } }. At best reasoning remains enabled; a strict gateway may reject the request beforerecoverJsonFromReasoningcan help. Please positively identify provider hosts and returnnullfor unknown endpoints, or add an explicit capability override. Add negative unknown-provider coverage and a SiliconFlow case: https://docs.siliconflow.cn/en/api-reference/chat-completions/chat-completions -
The Kimi matcher includes its own unsupported exception (
src/llm-client.ts:179-185). The comment sayskimi-k2.7-codecannot disable thinking, but/kimi.*k[12]|kimi.*thinking/imatches it and injectsthinking.type=disabled. Please use a disable-capable allowlist or add an explicit exclusion plus a negative test.
A clean way forward would be to narrow this PR to the live-verified DashScope DeepSeek V4 fix, then add the other families only with provider-specific evidence and tests.
Verification performed: full npm test passed on the PR branch; npm run build passed with no generated diff; the merge result with current master also built and passed the combined LLM API-key client tests.
Extends the single-line
/qwen3|deepseek.*r1|qwq/regex (issue #929) into a baseURL-aware rule table that injects the correct disable-thinking request parameter for each known Chinese reasoning-model family on its respective provider endpoint.Why a table (not just a regex fix)
The same model can require different parameter shapes on different providers. For example,
deepseek-v4-flashuseschat_template_kwargs.enable_thinking=falseon Aliyun DashScope butthinking.type="disabled"on DeepSeek's own API. Model name alone is not enough — the table selects the right parameter per(model family, provider).Coverage (parameters from each provider's official docs)
chat_template_kwargs.enable_thinking=falsechat_template_kwargs.enable_thinking=falsethinking.type="disabled"thinking.type="disabled"thinking.type="disabled"thinking.type="disabled"thinking_budget=0enable_thinking=falsethinking=falseAPI
buildDisableReasoningParams(model, baseURL)returns the correct params ornull.shouldDisableReasoningForJson(model)is kept as a backward-compatible boolean wrapper (used by existing tests).recoverJsonFromReasoning(PR fix(smart-extractor): only learn noise from a genuine empty extraction; recover JSON from the reasoning field #914) remains as a safety net for models/providers not yet in the table.Verification
Live test against DashScope
deepseek-v4-flashwith the extraction prompt, three variants:chat_template_kwargs.enable_thinking=falsethinking.type="disabled"Completion tokens -82%. DashScope
deepseek-v4-flashaccepts both parameters. Only this one model lane was live-verified; the rest are doc-sourced and rely onrecoverJsonFromReasoningas a safety net for unverified models/providers.Tests
test/llm-api-key-client.test.mjs— all 8 pass (6 original + 2 new: per-provider param table,deepseek-v4-flashend-to-end injection).Fixes #929