Skip to content

fix(lemonade): stop sending the dead LEMONADE_CTX_SIZE start variable - #6326

Open
kovtcharov-amd wants to merge 4 commits into
mainfrom
fix/3320-ctx-size-config
Open

kovtcharov-amd wants to merge 4 commits into
mainfrom
fix/3320-ctx-size-config

Conversation

@kovtcharov-amd

@kovtcharov-amd kovtcharov-amd commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

GAIA started Lemonade with LEMONADE_CTX_SIZE set and told users to set it too, but current Lemonade never reads that variable. So every auto-start and every "run this" hint that carried it appeared to set the context window and did nothing. This PR stops sending it. The window already reaches Lemonade through a channel that works: every GAIA model load sends ctx_size. The legacy CLI keeps its --ctx-size flag.

The issue suggested switching to lemonade config set ctx_size=N. I removed the variable instead, because that command persistently rewrites the user's own Lemonade config for every app on the machine. GAIA's per-load ctx_size already sets the window GAIA needs. The Go TUI had a copy of the same variable, so it is removed there too. Without that, the two front-ends would give different start commands.

Fixes #3320

🔍 Technical details
  • Evidence that the variable is dead: the issue's string scan of the 11.5.0 install found no reference to it. Upstream lemonade-sdk/lemonade names it only in test/server_cli2.py, which asserts that it is absent from the CLI help. config_file.cpp does not read it at v10.10.0, v11.0.0, or v2026.44.1 (the version GAIA pins). Context size is the ctx_size key in config.json, or /load ctx_size.
  • build_start_command returns an empty env for every modern form. describe_start_hint / render_command no longer print a LEMONADE_CTX_SIZE= prefix or a set "..." && form.
  • TUI: removed ctxPrefix, startEnv, ctxNote and the LEMONADE_CTX_SIZE override in profileCtxSize. The PowerShell variant is now emitted only when it differs from the cmd.exe line. The profile window is still used for the legacy flag and the loaded-window shortfall row.
  • The C++ client's context-overflow message no longer tells users to set the variable. It points only at start-lemonade.ps1 -CtxSize, which sends ctx_size on /load.
  • Tests that patched build_start_command and then asserted its own return value now call the real function. The env-merge and PATH-retention checks are kept, using a neutral variable.
  • No require_lemonade launch-ctx test was added. With nothing sent at launch, there is no launch-time claim left to verify. The /load ctx_size round trip is test(lemonade): the live /pull and /health tests cannot fail #3323's scope.

Test plan

  • python -m pytest tests/unit/test_lemonade_launcher.py tests/unit/test_lemonade_supervisor.py tests/unit/test_child_env_launch_sites.py tests/unit/test_lemonade_client_http.py tests/unit/test_llamacpp_backend.py tests/unit/test_ui_gpu_detection.py tests/unit/installer/test_init_ctx_size.py -q → 272 passed
  • cd tui && go vet ./internal/ui/preflight/ && go test ./... → all packages ok
  • black --check / isort --check-only clean on touched Python
  • npx vitest run src/components/__tests__/ConnectionBanner.test.tsx → 9 passed
  • CI build_tui.yml golangci-lint (not installed locally)
  • CI builds cpp/ (no C++ compiler on this machine; the change only edits two error-message string literals)

GAIA started modern Lemonade with LEMONADE_CTX_SIZE in its environment and
told users to set it, but no Lemonade release since 10.x reads it: the
server's env table has no such entry, and upstream only names it in a test
asserting it is absent from the CLI help. Every hint and auto-start that
carried it looked like it pinned the window and changed nothing.

The window already reaches Lemonade the way that works: every GAIA /load
sends ctx_size. So the dead channel is removed rather than replaced.

- build_start_command returns an empty env for every modern launch form;
  the legacy CLI keeps its --ctx-size flag.
- The Go TUI drops the same env prefix, the child-env injection, the
  LEMONADE_CTX_SIZE target override, and the remedy sentence telling users
  to set it in their service config.
- Tests that mocked build_start_command and then asserted their own
  fixture now call it for real; the env-merge and PATH-retention checks
  stay, against a neutral variable.

Fixes #3320
@github-actions github-actions Bot added devops DevOps/infrastructure changes llm LLM backend changes tests Test changes performance Performance-critical changes tui Go terminal UI (gaia-tui) labels Nov 1, 2026
@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Verdict: Approve with suggestions

GAIA used to start Lemonade with a context-size variable, and told users to set it, but current Lemonade never reads that variable. This PR stops sending it from the Python launcher and the Go TUI, so the start command and the hint shown to users are now just the bare command. The legacy --ctx-size flag is unchanged. The window still reaches the server, because every GAIA model load sends it. The code is clean and the tests now call the real start-command builder instead of a stub.

One spot outside the diff still gives the old advice. When the C++ client gets a "context too small" error, it tells the user to set the dead variable before restarting. Point it at the helper script or a reload instead, in this PR or a quick follow-up.

Real-world evidence

The evidence stage ran on ubuntu-latest at the PR head. A fake lemond stood in for Lemonade; it prints its argv and the variable, then exits 1. The results match the PR's claim:

  • CLI auto-start: the spawned child no longer gets the variable. Before this PR it got 65536.

    # PR head
    === GAIA daemon start: /tmp/fakelem/lemond ===
    lemond fake: args=[] LEMONADE_CTX_SIZE=[]
    # base (e28319e8d)
    === GAIA daemon start: LEMONADE_CTX_SIZE=65536 /tmp/fakelem/lemond ===
    lemond fake: args=[] LEMONADE_CTX_SIZE=[65536]
    
  • HTTP GET /api/system/status: 200 on both launcher types. The modern one returns the bare command, and the legacy one keeps its flag.

    { "start_instruction": "Run: /tmp/fakelem/lemond", "start_command": "/tmp/fakelem/lemond" }
    'start_command': '/tmp/fakelem/lemonade-server serve --ctx-size 32768'
    
  • TUI: go build ./... and go vet ./internal/ui/preflight/ both exit 0. The TUI was built but not launched.

  • Spot regression: /api/health and /api/sessions both return 200.

  • Agent UI rendered banner: pending the strix-halo lane. The underlying route evidence above is enough for merge-time CI.

🔍 Technical details

Issues found

🟡 The C++ client still tells users to set the dead variable (cpp/src/lemonade_client.cpp:324, :403)
On a context overflow, both error paths print set LEMONADE_CTX_SIZE=32768 in the environment before starting the server. This is the same kind of advice the PR removes elsewhere: it tells the user something worked when it did nothing. The second suggestion in that message, start-lemonade.ps1 -CtxSize 32768, does work, because the script sends ctx_size on /load (installer/scripts/start-lemonade.ps1:351). Drop the first line, at both sites:

                        "  Reload the model with a larger context via the helper script:\n" +
                        "    .\\installer\\scripts\\start-lemonade.ps1 -CtxSize 32768";

(Remove the Restart Lemonade with a larger context: and set LEMONADE_CTX_SIZE=… lines above it.)

🟢 Stale test fixtures still show the old command shape

  • src/gaia/apps/webui/src/components/__tests__/ConnectionBanner.test.tsx:17,57 still uses LEMONADE_CTX_SIZE=32768 /usr/bin/lemond as a sample start_command. The test still passes, but the fixture now shows output the backend can't produce. Use /usr/bin/lemond.
  • TestTheRunnableCheckLooksPastTheWindowPrefix (tui/internal/ui/preflight/lemonade_test.go:719) and stripEnvPrefix in check_test.go only cover the env-prefix forms this PR removes. They are harmless test helpers, so you can keep them as a guard or trim them.

Strengths

  • Tests that used to stub build_start_command and then check the stub's own return value now call the real function. The real-spawn supervisor test still proves the spec env is merged into the parent environment, using a neutral marker variable.
  • The TUI and Python changes match. profileCtxSize no longer honours a stray export, so a correctly loaded model won't be reported as short. The PowerShell variant is now emitted only when it differs from the cmd.exe line.
  • The rest of the docs were updated together: CLAUDE.md, the lemonade-specialist agent and the launcher module docstring.

@kovtcharov-amd

kovtcharov-amd commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator Author

Both suggestions are applied in the latest push.

  • On a context overflow, the C++ client no longer tells users to set the dead variable. At both sites it now points only at start-lemonade.ps1 -CtxSize, which sends ctx_size on /load.
  • The ConnectionBanner fixture uses /usr/bin/lemond. Vitest for that file passes (9/9).

I left stripEnvPrefix / firstWord and their test unchanged. They are test-only helpers, and they still parse the plain commands correctly; the prefix cases are now just unused coverage. No C++ compiler is available on this machine, so the C++ change (string literals only) relies on CI to build it.

@github-actions github-actions Bot added the cpp label Nov 1, 2026
@kovtcharov-amd

Copy link
Copy Markdown
Collaborator Author

This branch conflicts with main in one TUI file. It also overlaps #6317, which moves the same C++ context-overflow message into a shared decoder and still tells users to set LEMONADE_CTX_SIZE. The fix itself checks out: the context window still reaches Lemonade on every load, and the Python, Go and TUI tests show no regressions against main.

Next step: merge main and resolve the conflict as described below. Whichever of this PR and #6317 lands second also needs to drop the dead-variable line from the shared decoder.

🔍 Technical details
  • Conflict, tui/internal/ui/preflight/ctxsize.go (profileCtxSize). Take main's body (cfg := readDeviceConfig(p), the lemonade.RecommendedFor("local") NPU min-ctx loop, then the cfg.DefaultDevice == "npu" check) and drop only the ctxSizeEnv override block. Checked locally against origin/main@ba5db5bcf: go vet is clean, and go test ./internal/ui/preflight/ ./internal/lemonade/ passes.
  • fix(cpp): surface streamed server errors instead of "no tokens" #6317 interplay. fix(cpp): surface streamed server errors instead of "no tokens" #6317's describeServerError in cpp/src/lemonade_client.cpp still emits set LEMONADE_CTX_SIZE=32768 ..., and LemonadeErrorDecodeTest.StreamingSseErrorEventSurfacesContextRemedy asserts it with EXPECT_NE(msg.find("LEMONADE_CTX_SIZE"), npos). git merge-tree reports a conflict in that file between the two branches. Use this PR's wording in the decoder, and flip the assertion to check for start-lemonade.ps1.
  • Version check. Lemonade v10.7.0 did map LEMONADE_CTX_SIZE to --ctx-size in recipe_options.cpp. v10.8.0 through v10.10.0 only mention it in test/server_cli2.py. GAIA's floor is 10.2.0, but the per-load ctx_size covers 10.7.0 too, so nothing regresses.
  • Before/after (merged with main, ~6k tests, -n 8). No real regressions. Four test_model_slot_discipline failures under load pass 45/45 serially on both sides. The 5 removed Python tests and 14 removed Go tests all asserted the dead variable.

Resolves the overlap with #6317. The shared C++ error decoder it added now
gives this PR's context-overflow advice (reload via start-lemonade.ps1, no
LEMONADE_CTX_SIZE), and its test checks for that remedy and for the dead
variable's absence. profileCtxSize keeps main's NPU min-ctx lookup and drops
only the environment override.
@kovtcharov-amd

Copy link
Copy Markdown
Collaborator Author

This branch now merges cleanly with main and no longer conflicts with #6317. The shared C++ decoder #6317 added now gives this PR's advice for a context overflow: reload with start-lemonade.ps1, and don't set the dead variable. The Python, Go and C++ suites show no regressions against main.

🔍 Technical details
  • ctxsize.go: kept main's profileCtxSize body (the readDeviceConfig call, the NPU MinCtxSize loop and the npu check) and dropped only the LEMONADE_CTX_SIZE override. strconv is gone with it.
  • lemonade_client.cpp: both call sites use main's throwIfErrorBody. In describeServerError, the "Restart Lemonade… set LEMONADE_CTX_SIZE" lines are replaced by this PR's single helper-script line.
  • StreamingSseErrorEventSurfacesContextRemedy now asserts that start-lemonade.ps1 -CtxSize is present and LEMONADE_CTX_SIZE is absent.
  • Before/after, on main@bd99c0fd9 vs this branch merged with it:
    • C++ tests_mock (MSVC 2022): 1026/1026 pass on both sides.
    • Go, go test ./... in tui/: 2959 → 2957 pass, 0 failures. The removed tests asserted the dead variable. TestStartEnvLeavesVulkanCoopmatOn still holds without its test, because the child now inherits the parent env unchanged.
    • Python, 148 affected and neighbouring files: 4166 → 4170 pass. One parallel-run failure, TestSlowButAliveEmbeddedServer, passes 5/5 serially on both sides, so it's load flake. The 5 removed tests asserted the dead variable.
  • go vet ./..., black and isort are clean. Pylint's two E1101 hits (signal.SIGKILL, os.getuid) are Windows-only, show up on main too, and are in files this PR doesn't touch.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cpp devops DevOps/infrastructure changes llm LLM backend changes performance Performance-critical changes tests Test changes tui Go terminal UI (gaia-tui)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(lemonade): LEMONADE_CTX_SIZE is read by nothing — the auto-start ctx fix (#839) is inert

1 participant