Skip to content

[Feat]: Per-agent chatModel priority + fallback chain via YAML config #37

Description

@FireTable

Problem

Today every LangGraph node resolves its chat model from a single global registry (backend/model.ts:getChatModel()lib/provider/model-registry.ts:getChatModelFromDB()) that round-robins evenly across every enabled (provider, model, key) tuple. There is no way to say:

  • "weather-agent should prefer minimax/MiniMax-M3, then fall back to openai/gpt-4o"
  • "code-agent should use Anthropic Claude Opus, never round-robin into a smaller model"

docs/PROVIDERS.md already calls out per-agent binding as future direction, and the recent round-robin refactor (#36) explicitly removed Runnable.withFallbacks because it returned a Runnable (not BaseChatModel) and dropped .bindTools / .withStructuredOutput for downstream node consumers. So we need a per-agent fallback mechanism that does not wrap the chat model in a different Runnable type.

Proposal

Add a YAML config file (path TBD — likely config/agents.yaml, mirrored to /app/config/agents.yaml in the Docker image) that maps each agent to an ordered list of chat-model candidates plus a fallback policy.

Sketch:

agents:
  weather:
    priority:
      - { provider: minimax, model: MiniMax-M3 }
      - { provider: openai,  model: gpt-4o-mini }
    fallback: sequential        # try each in order on error

  code:
    priority:
      - { provider: anthropic, model: claude-opus-4-8 }
    fallback: none               # fail loud if the primary is down

  router:
    priority:
      - { provider: minimax, model: MiniMax-M3 }
      - { provider: openai,  model: gpt-4o-mini }
      - { provider: groq,    model: llama-3.3-70b-versatile }
    fallback: sequential

Resolution path:

  1. getChatModel({ agentName }) reads the YAML once at startup (or on first call), caches in the same LRU shape already used for the tuple list.
  2. For each candidate, looks up the (provider, model) in the existing provider table — reuse getChatModelFromDB's decrypt + ChatOpenAI build, do not fork the model-construction path.
  3. Walks the priority list and calls the model directly. No Runnable.withFallbacks — iterate manually inside getChatModel, catch and advance on thrown error.
  4. On final exhaustion, fall through to the existing env-only ChatOpenAI so a fresh checkout still boots before any provider is seeded.

Validation behavior (must)

If the YAML references an agentName that no LangGraph node exists for, skip that entry with a console warning at load time — do not crash the boot. This matters because adding/removing a sub-agent today requires touching state.ts, router-agent-node.ts, AND agent.ts routeToSubAgent union (see [[router-decision-schema-duplicated]]); the YAML should not make that worse.

If a candidate references a (provider, model) that does not exist in the DB registry, skip that candidate at resolve time, advance to the next. Log a warning with the offending key so the operator notices in logs.

If the entire priority list resolves to zero usable candidates (all invalid), fall through to the env fallback silently — never throw on a config mistake in the hot path. Crash only on YAML parse errors, which are a real config bug and should fail boot.

Acceptance criteria

  • config/agents.yaml exists with sensible defaults (current round-robin behavior expressed as a YAML fallback chain, or empty file → existing behavior unchanged).
  • getChatModel({ agentName }) returns the first usable candidate per the YAML.
  • On a thrown error from candidate N, candidate N+1 is tried; the original error is only re-thrown when the list is exhausted (and only after the env fallback).
  • Invalid agentName or (provider, model) entries are skipped, not fatal; both paths produce a structured console.warn with enough context to grep.
  • YAML parse error → boot fails loudly with file path + line.
  • backend/agent/{chat,code,weather,crypto}-agent.ts + backend/node/{rename-thread-agent,thread-summarize,router-agent}-node.ts all pass agentName to getChatModel.
  • docs/PROVIDERS.md updated with the new file, schema, and resolution path. docs/APIS.md unchanged (no API surface).
  • Tests: YAML parse happy/sad paths; per-agent candidate selection; skip-invalid behavior; fallback exhaustion → env fallback; manual-iteration fallback does NOT wrap the model in Runnable.withFallbacks (regression guard for [Refactor]: Round-robin provider registry without Runnable.withFallbacks chain #36).

Out of scope

  • Cross-process YAML hot-reload (mirror the existing 60s LRU + admin invalidation pattern).
  • Per-candidate retry budgets / circuit breakers (single attempt per candidate, fail-fast to next).
  • Adding a YAML editor to the admin UI — file-based only for v1.
  • Health-based skipping (provider.last_error_at cooldown) — separate issue if/when we add health columns.

Why this is more than "just write YAML"

The hard part is keeping the fallback chain type-compatible with the existing consumers (ChatOpenAI.bindTools(...) in every node). The previous Runnable.withFallbacks approach (#36) silently broke that. The manual try/catch loop in getChatModel is intentional and the regression test in the AC list is non-negotiable.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions