refactor: replace monkey-patching with dependency injection in train() — closes #64#66
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors the --dry-run mechanism to use dependency injection instead of global monkey-patching, introducing a _StubAdapter to safely mock environments that bypass _make_llm_client (such as taubench, entropic, and openclaw). It also adds up-front validation for taubench configurations and includes comprehensive tests to verify dry-run behavior and configuration validation. The review feedback correctly highlights that passing None as the make_llm_client argument in several tests violates the expected non-optional LLMClientFactory type signature, and suggests replacing None with a dummy lambda to maintain type safety.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
…ganthos#64 `train()` now accepts keyword-only `make_llm_client` and `env_builders` parameters; the CLI dry-run path builds them as closures over the user's config and passes them in directly. This removes the module-level mutation of `clawloop.train._make_llm_client` and `ENV_BUILDERS` that PR aganthos#62 introduced, and the `_DryRunLLMClientConfig` subclass that PR aganthos#65 added to identify roles for the monkey-patched factory. Changes ------- - `clawloop/train.py`: - `train()` signature gains `*, make_llm_client=None, env_builders=None`; defaults preserve current behavior. - All six `_build_*` helpers gain `make_llm_client` as a third positional parameter (uniform signature). Only `_build_math` actually uses it today; others accept it for the future refactor tracked by the out-of-scope notes in the plan. - New module-level alias `LLMClientFactory = Callable[[LLMClientConfig], Any]` for readable type hints. - `clawloop/cli.py`: - Delete `_DryRunLLMClientConfig` and the `LLMClientConfig` import that only existed to subclass it. - Replace `_install_dry_run_clients` (which mutated module state) with `_build_dry_run_overrides(config)` (pure: returns `(factory, builders)`). - `cmd_run` passes the overrides into `train(...)` directly. - `_StubAdapter` and `ENVS_USING_MAKE_LLM_CLIENT` are unchanged. - Tests: - `tests/test_train_config.py::TestTrainEndToEnd::test_harness_learning_math`: drop `with patch("clawloop.train._make_llm_client")` and pass `make_llm_client=_pick_client` to `train()` directly. - `tests/test_cli.py`: - Remove the subclass round-trip test (subclass is gone). - Replace with `test_dry_run_overrides_map_roles_correctly` and `test_dry_run_overrides_stub_bypassing_env` exercising `_build_dry_run_overrides` directly. - Add `test_dry_run_does_not_mutate_module_state` — invokes `cmd_run` with `dry_run=True` and asserts `clawloop.train._make_llm_client` and `ENV_BUILDERS` are unchanged after the call. Locks in the no-global-mutation contract. - Keep `test_public_llm_client_config_schema_excludes_dry_run_vocab`. - `tests/unit/test_train.py`: builder invocations now pass `None` as the third positional arg (openspiel doesn't use it). Verification ------------ - pre-commit clean (ruff + ruff-format) - pytest tests/test_cli.py tests/test_train_config.py tests/unit/test_train.py → 68 pass - Full suite: 1046 passed, 49 skipped, 0 failed - `clawloop run examples/configs/math_harness.json --dry-run` exits 0 - `clawloop run examples/configs/taubench_harness.json --dry-run` with all API key env vars unset exits 0 - `clawloop eval` still exits 2 with the redirect Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per gemini-code-assist review on PR aganthos#66: passing `None` for the `make_llm_client` parameter violates the declared type signature `LLMClientFactory = Callable[[LLMClientConfig], Any]`. The builders we invoke (openspiel, taubench stub) never call the factory, but the type contract still applies. Replace each `None` with `lambda _: None` — a callable that ignores its arg and returns None. Satisfies the type signature; behavior is unchanged because the callees don't invoke it. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
82df5aa to
7d8e894
Compare
Summary
Closes #64.
train()now accepts keyword-onlymake_llm_clientandenv_buildersparameters; the CLI dry-run path builds them as closures over the user's config and passes them in directly totrain(...). This eliminates two pieces of implicit global state that the prior dry-run wiring had introduced:_train._make_llm_client = _mock_make(from PR fix: address PR #60 review comments on clawloop run --dry-run + taubench #62) — module-level reassignment._train.ENV_BUILDERS[env_type] = stub_builder(from PR fix: address PR #60 review comments on clawloop run --dry-run + taubench #62) — dict mutation.The
_DryRunLLMClientConfigsubclass added in PR #65 also goes away: with DI the role lookup lives in a closure created fresh pertrain()call, so no cfg tag is needed at all.Why this is a real improvement, not just a refactor
The previous design had implicit contracts that DI removes:
_make_llm_clientbeing a module-level reassignable function. Iftrain.pyever inlined it, moved it into a class, or decorated it, dry-run would silently stop mocking. New testtest_dry_run_does_not_mutate_module_stateasserts the no-mutation contract directly.ENV_BUILDERSwas mutated globally, so the change persisted for the process lifetime — paralleltrain()calls would have raced.with patch(\"clawloop.train._make_llm_client\")— unnecessary ceremony with DI. TheTestTrainEndToEndtest in this PR drops the patch and passesmake_llm_client=_pick_clientdirectly.What changed
clawloop/train.pytrain()gains*, make_llm_client=None, env_builders=None(defaults preserve current behavior). All six_build_*helpers acceptmake_llm_clientas a third positional parameter — only_build_mathuses it today; others accept it for signature uniformity and future use. AddedLLMClientFactory = Callable[[LLMClientConfig], Any]type alias.clawloop/cli.py_DryRunLLMClientConfigand itsLLMClientConfigimport. Replaced_install_dry_run_clients(mutates module state) with_build_dry_run_overrides(config)(pure: returns(factory, builders)).cmd_runpasses overrides intotrain(...)._StubAdapterandENVS_USING_MAKE_LLM_CLIENTunchanged.tests/test_train_config.pyTestTrainEndToEnd::test_harness_learning_mathdropswith patch(...)ceremony for direct DI.tests/test_cli.pytest_dry_run_overrides_map_roles_correctly,test_dry_run_overrides_stub_bypassing_env,test_dry_run_does_not_mutate_module_state. Kept the schema-cleanliness test.tests/unit/test_train.pyNoneas the third positional arg.PR layering
This builds on #62 and #65 (both still open at time of writing). The diff against
mainwill include all three PRs' commits until #62 and #65 merge. Suggested merge order: #62 → #65 → #64. Each PR's diff against the post-merge main is a clean forward-looking step.If you'd rather skip #65 (since DI here removes both the field from #62 and the subclass from #65 anyway), you could merge #62 → #64 and close #65. Either path is reachable.
Test plan
pre-commit run --all-filesclean (ruff + ruff-format)pytest tests/test_cli.py tests/test_train_config.py tests/unit/test_train.py→ 68 passclawloop run examples/configs/math_harness.json --dry-runexits 0 (realMathAdapter+ injected mock factory)clawloop run examples/configs/taubench_harness.json --dry-runwithOPENAI_API_KEY/ANTHROPIC_API_KEY/GEMINI_API_KEYunset exits 0 (stub adapter via injectedenv_buildersoverride)clawloop evalstill exits 2 with the truthful redirecttest_dry_run_does_not_mutate_module_statedirectly assertsclawloop.train._make_llm_clientandENV_BUILDERSare unchanged after a dry-runOut of scope (future follow-ups)
_build_entropic,_build_openclaw,_build_taubench,_build_openspielto use the injectedmake_llm_clientfor their internal LLM construction. They accept the parameter for signature uniformity but still drive LLMs internally. Routing them throughmake_llm_clientwould let dry-run drop_StubAdapterfor those envs and exercise the real adapter logic with mocks.make_llm_clientthroughReflector,LocalEvolver, or other harness internals.🤖 Generated with Claude Code