You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Today every LangGraph node resolves its chat model from a single global registry (backend/model.ts:getChatModel() → lib/provider/model-registry.ts:getChatModelFromDB()) that round-robins evenly across every enabled (provider, model, key) tuple. There is no way to say:
"weather-agent should prefer minimax/MiniMax-M3, then fall back to openai/gpt-4o"
"code-agent should use Anthropic Claude Opus, never round-robin into a smaller model"
docs/PROVIDERS.md already calls out per-agent binding as future direction, and the recent round-robin refactor (#36) explicitly removed Runnable.withFallbacks because it returned a Runnable (not BaseChatModel) and dropped .bindTools / .withStructuredOutput for downstream node consumers. So we need a per-agent fallback mechanism that does not wrap the chat model in a different Runnable type.
Proposal
Add a YAML config file (path TBD — likely config/agents.yaml, mirrored to /app/config/agents.yaml in the Docker image) that maps each agent to an ordered list of chat-model candidates plus a fallback policy.
Sketch:
agents:
weather:
priority:
- { provider: minimax, model: MiniMax-M3 }
- { provider: openai, model: gpt-4o-mini }fallback: sequential # try each in order on errorcode:
priority:
- { provider: anthropic, model: claude-opus-4-8 }fallback: none # fail loud if the primary is downrouter:
priority:
- { provider: minimax, model: MiniMax-M3 }
- { provider: openai, model: gpt-4o-mini }
- { provider: groq, model: llama-3.3-70b-versatile }fallback: sequential
Resolution path:
getChatModel({ agentName }) reads the YAML once at startup (or on first call), caches in the same LRU shape already used for the tuple list.
For each candidate, looks up the (provider, model) in the existing provider table — reuse getChatModelFromDB's decrypt + ChatOpenAI build, do not fork the model-construction path.
Walks the priority list and calls the model directly. No Runnable.withFallbacks — iterate manually inside getChatModel, catch and advance on thrown error.
On final exhaustion, fall through to the existing env-only ChatOpenAI so a fresh checkout still boots before any provider is seeded.
Validation behavior (must)
If the YAML references an agentName that no LangGraph node exists for, skip that entry with a console warning at load time — do not crash the boot. This matters because adding/removing a sub-agent today requires touching state.ts, router-agent-node.ts, AND agent.ts routeToSubAgent union (see [[router-decision-schema-duplicated]]); the YAML should not make that worse.
If a candidate references a (provider, model) that does not exist in the DB registry, skip that candidate at resolve time, advance to the next. Log a warning with the offending key so the operator notices in logs.
If the entire priority list resolves to zero usable candidates (all invalid), fall through to the env fallback silently — never throw on a config mistake in the hot path. Crash only on YAML parse errors, which are a real config bug and should fail boot.
Acceptance criteria
config/agents.yaml exists with sensible defaults (current round-robin behavior expressed as a YAML fallback chain, or empty file → existing behavior unchanged).
getChatModel({ agentName }) returns the first usable candidate per the YAML.
On a thrown error from candidate N, candidate N+1 is tried; the original error is only re-thrown when the list is exhausted (and only after the env fallback).
Invalid agentName or (provider, model) entries are skipped, not fatal; both paths produce a structured console.warn with enough context to grep.
backend/agent/{chat,code,weather,crypto}-agent.ts + backend/node/{rename-thread-agent,thread-summarize,router-agent}-node.ts all pass agentName to getChatModel.
docs/PROVIDERS.md updated with the new file, schema, and resolution path. docs/APIS.md unchanged (no API surface).
Per-candidate retry budgets / circuit breakers (single attempt per candidate, fail-fast to next).
Adding a YAML editor to the admin UI — file-based only for v1.
Health-based skipping (provider.last_error_at cooldown) — separate issue if/when we add health columns.
Why this is more than "just write YAML"
The hard part is keeping the fallback chain type-compatible with the existing consumers (ChatOpenAI.bindTools(...) in every node). The previous Runnable.withFallbacks approach (#36) silently broke that. The manual try/catch loop in getChatModel is intentional and the regression test in the AC list is non-negotiable.
Problem
Today every LangGraph node resolves its chat model from a single global registry (
backend/model.ts:getChatModel()→lib/provider/model-registry.ts:getChatModelFromDB()) that round-robins evenly across every enabled(provider, model, key)tuple. There is no way to say:weather-agentshould preferminimax/MiniMax-M3, then fall back toopenai/gpt-4o"code-agentshould use Anthropic Claude Opus, never round-robin into a smaller model"docs/PROVIDERS.mdalready calls out per-agent binding as future direction, and the recent round-robin refactor (#36) explicitly removedRunnable.withFallbacksbecause it returned aRunnable(notBaseChatModel) and dropped.bindTools/.withStructuredOutputfor downstream node consumers. So we need a per-agent fallback mechanism that does not wrap the chat model in a different Runnable type.Proposal
Add a YAML config file (path TBD — likely
config/agents.yaml, mirrored to/app/config/agents.yamlin the Docker image) that maps each agent to an ordered list of chat-model candidates plus a fallback policy.Sketch:
Resolution path:
getChatModel({ agentName })reads the YAML once at startup (or on first call), caches in the same LRU shape already used for the tuple list.(provider, model)in the existingprovidertable — reusegetChatModelFromDB's decrypt +ChatOpenAIbuild, do not fork the model-construction path.Runnable.withFallbacks— iterate manually insidegetChatModel, catch and advance on thrown error.ChatOpenAIso a fresh checkout still boots before any provider is seeded.Validation behavior (must)
If the YAML references an
agentNamethat no LangGraph node exists for, skip that entry with a console warning at load time — do not crash the boot. This matters because adding/removing a sub-agent today requires touchingstate.ts,router-agent-node.ts, ANDagent.tsrouteToSubAgent union (see[[router-decision-schema-duplicated]]); the YAML should not make that worse.If a candidate references a
(provider, model)that does not exist in the DB registry, skip that candidate at resolve time, advance to the next. Log a warning with the offending key so the operator notices in logs.If the entire priority list resolves to zero usable candidates (all invalid), fall through to the env fallback silently — never throw on a config mistake in the hot path. Crash only on YAML parse errors, which are a real config bug and should fail boot.
Acceptance criteria
config/agents.yamlexists with sensible defaults (current round-robin behavior expressed as a YAML fallback chain, or empty file → existing behavior unchanged).getChatModel({ agentName })returns the first usable candidate per the YAML.agentNameor(provider, model)entries are skipped, not fatal; both paths produce a structuredconsole.warnwith enough context to grep.backend/agent/{chat,code,weather,crypto}-agent.ts+backend/node/{rename-thread-agent,thread-summarize,router-agent}-node.tsall passagentNametogetChatModel.docs/PROVIDERS.mdupdated with the new file, schema, and resolution path.docs/APIS.mdunchanged (no API surface).Runnable.withFallbacks(regression guard for [Refactor]: Round-robin provider registry without Runnable.withFallbacks chain #36).Out of scope
provider.last_error_atcooldown) — separate issue if/when we add health columns.Why this is more than "just write YAML"
The hard part is keeping the fallback chain type-compatible with the existing consumers (
ChatOpenAI.bindTools(...)in every node). The previousRunnable.withFallbacksapproach (#36) silently broke that. The manual try/catch loop ingetChatModelis intentional and the regression test in the AC list is non-negotiable.