Genesis voice, live build and research missions, merged with the configuration repository - #8
Merged
Conversation
On Railway STUDIO_ANALYSIS_USD is 0, so the reasoning review never ran — not for want of a credential, which was there all along. Yesterday's change then hid the manual button for that reason too, which was wrong: `review` pays from the run's own ceiling, not from STUDIO_ANALYSIS_USD, so a person can still ask. The server now says whether asking by hand could work. Nothing to interpret, no credential and a ledger that cannot cover the ceiling are walls and hide the button; the automatic pass being off is not, and keeps it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…browser checks match the pages The preview entry pinned 8766, so a stale fixture server on that port stopped the Studio from starting at all. The port now comes from PORT when it is set, and the launch entry may move. Two browser checks asserted against pages that had moved on: the report no longer carries an audience switch in its address, and the Genesis configuration page says weekly allowance. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ve the history The Runs page printed the raw arm id whenever nothing had named the competitor, so a scripted check read `oracle` in the row and in the Setups block. An unnamed id now goes through the same resolver the rest of the Studio uses, and the job payload carries one named arm per competitor, so the history row, the expansion, the CSV export and the report agree on what a setup is called. A benchmark Studio already refuses to launch the scripted checks. It no longer lists old runs made of them either; the fixture Studio keeps them, since they are all it has. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A round had two reports. `audiences.yaml` decided which competitors a reader was allowed to see, the Studio had a Public/Internal toggle, and the same run rendered differently depending on who asked. That is gone. Every competitor that ran appears in the report, with its exact cost, and every reader sees the same page. What went, on both surfaces: - `wb_report/audiences.yaml`, and with it `load_audiences`, `gate_arms`, `is_lab` and the GateError that refused to render a lab competitor outside the internal audience. - The cost columns that showed ratios instead of dollars, the withheld-competitor count, the DO NOT EXPORT watermark, and the audience in the markdown header. - `Plan.audience`, the `audiences=` argument on `resolve()` and its unknown-audience check; both `--audience` flags on `wb report` and `wb summary`. - In the Studio, `visible_setups`, the `?audience=` plumbing, the toggle and the internal band; `without_reasoning`, so a reader keeps the provider's reasoning summaries the timeline always had. Reports are filed by run alone now: `report-<run_id>.md` and `.html`, `summary-<timestamp>.html`. Display yes, export no. A lab build is shown like any other competitor, and the interface refuses to export or print a report that carries one, so the rule the retired audiences file stated - that a lab build never reaches an exportable artifact - still holds where it mattered. `audience` stays an accepted and ignored plan key, the treatment `approved_by` already has, so an older plan file still loads. It also stays in the config hash as the constant every plan carried: it chose how a round was rendered, never what was measured, so a stored run stays resumable and comparable instead of looking like a different measurement. Both pinned hashes are unchanged, which is the evidence for that claim. This retires rule 10 of PLAN.md section 1.1, "audience rules are code", on Lucas's decision of 11 September. The constitution's section III still asks for Carlos's explicit say-so before a fixed rule is reopened, and that has not been recorded. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… was missing 350ffb3 staged wb_studio/app.py and wb_orchestrator/cli.py whole, which carried part of feature 024's gate work into HEAD without the modules it calls. HEAD's app.py imported front_door_path and front_door_secret from monarch_setup and background_wanted from genesis_autonomy, none of which existed there, so the Studio could not import at all; cli.py's budget acknowledge called a ledger method that was not in budget.py. This commits the other half. No blame in that: the two files could not be split without leaving HEAD broken either way. The gates themselves, from specs/024-architecture-search (FR-001 to FR-009): - The front door relayed with no credentials on every method, and do_PUT, do_PATCH and do_DELETE never called authorised() at all, so anyone holding the hosted address could write into a running attempt's world and the approval rule would read that write as the competitor failing. Basic Auth cannot gate a path the competitor under test calls, so the door now wants a secret segment that rides in the seed URL. Monarch sends nothing new. It fails closed: with no secret set, the door serves nothing. - Every autonomy dial now starts off, and the watcher and the scheduler start only once a person turns one on. A fresh workspace with provider keys used to begin working cards within thirty seconds and could dispatch a paid run whose operator was genesis:smoke and whose approver was a model turn. - Pause is read in Genesis.chat, the one funnel every paid turn passes through, and in Scheduler.run_due, so all six daily jobs stop instead of spending through it. A skipped job is recorded and not stamped, so it runs when the pause lifts rather than being lost for the day. - The day's allowance counts the reserved maximum of every run Genesis launches. It summed only its own turns, so N self-launched runs each saw the same untouched allowance. - A ledger line now carries which allowance it draws on, separately from who asked for it. usage.who maps a conversation turn's by='person' to 'Studio user', so a person's Genesis turn was budgeted as a benchmark round and the research allowance never saw it. Attribution is a display fact; accounting is a money fact. - An overrun still blocks every admission, because a provider that charged above what it promised is exactly when to stop and look. What it no longer requires is editing research/budget.sqlite3 by hand: a named person acknowledges one overrun with a reason, recorded beside it, and only that one stops blocking. No money changes and nothing leaves the record. - A paid launch from the Studio names the person who asked for it, and refuses an agent presenting itself as one. The approval-record half of D5 is not done. - One attempt can no longer spend the whole round: every arm takes an attempt cap instead of the run ceiling. The cap is four times an even split, clamped to the ceiling and never below what a single request reserves — a three attempt round sized for three expensive requests has nothing to cut, and capping under one request's price refuses every attempt instead of limiting it. - Genesis cannot fetch the inside of the network it runs in. fetch_source validated only the spelling of a URL, so a dropped link to the cloud metadata endpoint was read and stored. The check is on the resolved address. tests/conftest.py sets WB_OPERATOR for the suite: thirty-five tests create paid jobs and the requirement is the product's, not theirs. test_studio_launch_gates deletes it and asserts the refusal. 88 tests across seven files; full suite running at the time of writing, one failure known to predate this work (test_run_page.py::test_index_has_evidence_tabs_and_runs_table_counts). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… behind 350ffb3 committed reports.js and report.css but not charts.css, so the markup for the failure-mode table went in without a single one of its rules. The committed reports.js emits `chart-matrix fail-modes`, `mode-head`, `mode`, `count` and a `<tfoot>`; the number of rules for any of those at HEAD was zero. Nothing was broken in the working tree, which had the file, so the gap was invisible until someone checked the commit out clean. Measured in the fixture Studio by deleting exactly those nine rules from the live stylesheet: the heat tint went to transparent, cell padding fell from 6px 10px to the browser's 1px, the numerals lost their centring and rendered in Newsreader instead of IBM Plex Mono, and the foot lost its rule. The figure the "Where it failed" section is built around read as cramped serif prose. The rules themselves are three steps of one hue, cut against the heaviest cell in the table. The tint carries the signal and the type stays ink, so light and dark both clear contrast from the same rule with no per-theme override. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…module it imports `wb_studio/genesis_critic.py` imports `genesis_engineer` at module level, and that file is not in the repository. Staging the critic on its own put a file into the tree that raises on import. Nothing imports it, so the break was latent rather than fatal, but a committed file that cannot be imported is still a defect. It returns to untracked with its working copy intact, and belongs in the commit that carries genesis_engineer.py and the rest of its wiring. The two lines this session changed in it - reading the one report, and the docstring that went with it - are preserved on disk and will ride along then. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…t never carried `wb_studio/allowances.py` was untracked while six places in HEAD imported it: wb_studio/app.py in four (from 350ffb3), wb_studio/genesis.py in two, and three test files. app.py's are inside handlers, so on a clean checkout the Studio started and then raised the first time anyone opened the Budget page or set an allowance. My 41d4cde added tests/test_genesis_allowance.py, which imports it at module level and turned that latent failure into a collection error — louder, and the reason it was found. I own the noise, not the gap: the dependency has been dangling since 350ffb3, and `git diff HEAD -- <path>` reports nothing for an untracked file, which is how I read the module as committed when I checked. `git ls-files --error-unmatch` is the check that answers the question actually being asked. The module is the weekly allowance inside the one lab week: a named ceiling per kind of work (genesis, analysis, rounds) that a reservation passes before it is admitted, never a second pot of money. Its test comes with it, 30 tests green with tests/test_genesis_allowance.py. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four half-commits shipped on 11 September within an hour, three of them mine. The shape is always the same: a tracked file starts importing a module that has not been added, every test passes because the module is sitting in the author's working tree, and the commit puts code into the repository that raises on a clean checkout. One of them was a Studio that started and then failed the first time anyone opened the Budget page. No test can catch this, because the tests run where the file exists. The check reads the source on disk but resolves every import against `git ls-files`, so a module nobody added is invisible to it - which is the question being asked. It needs no clean checkout and takes under a second. Two traps, both of which produced a scan that reassured instead of checking, and both of which we walked into before the check worked: - Indented imports count. The Studio imports lazily inside functions all over; anchoring at `^` misses 718 of the 1151 wb_* imports in the tree. - `from wb_studio import allowances` names a module too. Reading only the part after `from` resolves `wb_studio`, which always exists, and never looks at `allowances` - exactly the import that broke the Budget page. An imported lowercase name is only reported when the package neither tracks it as a submodule nor defines it as a symbol, so ordinary `from x import helper` stays quiet. Runs before the tests, since it explains failures the tests would report as import errors. Verified both ways: clean on a worktree of HEAD, and it fails on a tracked file importing an untracked module. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It said "Each of these is committed code that raises on a clean checkout." Run locally, which is where it is most useful, the flagged imports usually are not committed yet - that is the whole point of catching them there. The message claimed a present-tense break that had not happened, in the one line whose job is to stop someone shipping one. It now says what is actually true: each raises on a clean checkout once committed. Also drops a stale sentence from the report tests, which still described reports as hiding lab setups from the public view. Reported by ailabs-3b. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three people wrote this check on the same afternoon and each shipped a different structural blindness: an anchor at `^` that missed indented imports, a match on the dotted path alone that could not see `from wb_studio import allowances`, and a name capture that read `genesis_engineer as engineer` as one token. Every version would have passed the bug it was written to catch. The first two were already in the docstring. The third was not, and it is the alias behind one of this morning's four half-commits, so it belongs beside them rather than in a thread nobody will read again. Suggested by ailabs-34, who found it in their own version. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…periment Brainstorm, spec, plan and tasks for feature 024, from Lucas's decisions of 11 September. The design of record carries the nine of them; the short version is decision 7: the architecture editor is the destination, not a model. If a composed architecture beats Monarch Enterprise on the frozen set, that composition is the proposal. The lab is not running a leaderboard with an assistant attached — it is running architecture search over Monarch's design space, scored by the benchmark. What that costs, and why most of this feature is repair rather than construction. Search finds whatever the scorer rewards, so the scorer has to be worth optimising against: - There is no development / held-out split. AI-LABS-DIRECTION.md asks for one three times and no implementation exists, so every win the loop reports would be unfalsifiable. - genesis_hypotheses.smallest_plan sizes an experiment with a two-proportion normal approximation floored at ten tasks, while settle() and the round report decide it with a paired sign test. Below six discordant pairs the sign test cannot reach p<0.05 at any win count, and ten tasks typically yield three or four. The lab's sizing function recommends experiments its own settling function can never conclude. - The fitness function is the break-even curve — configure once, execute cheaply, against a harness paying full price every run — not the pass rate. Scored on accuracy alone the search finds an architecture that wins by spending more, which is the opposite of the claim. Phase 0 found that six of the nine capability requirements are wiring of modules that already exist: per-phase cost is already in langfuse_cost.CostSummary.by_phase and discarded below the arm; the strict cohort key and exclusion_reason are written and unused in leaderboard.py; run-only execution is built and only lacks recipe data; the experiment record is the hypothesis record needing four fields, not a second store. Two things are genuinely new: a secret segment on the front-door proxy, and one timestamp at the authoring boundary. User Story 1, the gates, is implemented and committed (41d4cde, 42b639a, plus the approvals wiring done by other sessions the same day). US2 to US5 are specified and not started. tasks.md records what is done, what is not, and four deliberate deviations from the plan with the reason for each. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…t orphaned Two things, both found by the check catching its own author. **The scan could not see a module named as a string.** `scheduler.MODULES` and `genesis_plugins.MODULES` are tuples of dotted names resolved through importlib, and HEAD names `wb_studio.genesis_engineer` and `wb_studio.genesis_critic` in both while carrying neither. `discover()` catches ImportError and moves on, so a fresh checkout loses two daily jobs in silence - failing open, which is worse than the error it replaces. String literals shaped like a module are now resolved too, reported separately because the consequence is different. A dotted string is only treated as a module when its parent is a package, so `wb_studio.report_data.run_report` stays quiet; tests are exempt, since naming `wb_studio.not_there` is how the skip path is proved. Reported by ailabs-34, who checked a case they already knew the answer to. **And the new check immediately caught me doing it again.** 3fdec18 swept in another session's new test for `resolve_service` while `wb_studio/reports.py`, which defines it, stayed out - a test that fails on a clean checkout, the fourth time this shape has shipped today and the second from my hands. reports.py holds sixty lines of someone's unfinished work and is not mine to commit, so the test comes back out of the tree instead. Both halves stay on disk, together, for whoever owns them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e modules HEAD already named scheduler.MODULES and genesis_plugins.MODULES have named `wb_studio.genesis_engineer` and `wb_studio.genesis_critic` since 350ffb3, while carrying neither. `discover()` catches ImportError and moves on, so a clean checkout lost two daily jobs in silence, with nothing anywhere saying so. This carries the modules those names point at, and CI is green on the pair again. The work is ailabs-34's, staged here at Lucas's direction because the wiring had already been swept into HEAD by my earlier whole-file commits and only its other half was missing. Eighteen files in one commit because they do not work apart: genesis_critic imports genesis_engineer, and genesis_plugins registers all three. The engineer loop: at 08:00 São Paulo it reads the runs it has not settled, computes their failure buckets for free, and asks for one typed spec on the worst bucket that points at code. A Codex agent implements it in a throwaway detached worktree; the Studio runs the verify command itself and files a card with the diff, the test result and the cost. Nothing is applied, merged or pushed, and the `engineer` dial starts off. I read genesis_engineer.py before staging it, since it shells out to Codex with workspace-write. The controls hold: the verify command is a fullmatch against a two-entry allowlist rather than a string a model chose, the verdict is the Studio's own exit status and never the agent's account of it, the ledger reserves before the agent starts and holds the ceiling when usage cannot be read, and the worktree is removed in a finally. One thing for a person to weigh rather than a defect: the allowlist admits `..` in the pytest path, so the command can name a directory outside the worktree it runs in. 108 passed across the genesis engineer, critic, figures, images, loop, repairs, layer0 and patch suites, the report lab gate and the static CSP check. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ten to stop
`[\w./-]{1,120}` after `pytest` was not a path constraint. It admitted an
absolute path, the worktree's parent, an escape through `..`, and `--pdb`, which
is not a path at all. pytest executes conftest.py and every module it collects,
so each of those is code execution on the lab's machine at a location a model
chose - through the one gate whose docstring says the allowlist exists precisely
so a model cannot choose what the lab runs.
The only thing stopping `../../../Users/<name>/...` was the space in the path.
An accidental guard is not a guard.
The path is now anchored at `tests/` and `..` is refused outright; rejecting `..`
alone would have left the absolute path and the bare flag. `node
tests/browser/suite.cjs --only runs` is refused too, as it should always have
been: that suite writes the review snapshots and the critic must not be able to
aim it. The refusal sentence and the spec prompt both describe the shape, so a
model that gets it wrong is told how rather than only told no.
Found and fixed by ailabs-34 after I flagged `..` as a judgement call in
267e4de's message. That framing was wrong - I had checked the escape I thought of
and not the class, and three of the four cases were ones I never tried. Verified
here against the committed regex and the fix: all four rejected, and
`tests/<path>`, `tests/<path> -q` and the bare browser suite still admitted.
117 passed, with thirteen allowlist cases where there were six.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… the run reading runs again Two things were asked for and one was already there. **The reasoning review on Railway was never broken by a missing credential.** Both Gemini keys are set; `STUDIO_ANALYSIS_USD` was `0`, so `schedule_narrative` recorded "the automatic reading is off" and skipped it. The variable is now 0.50 on Railway — the worst case for one reading is $0.355 at the hard payload cap. Yesterday's change also hid the manual button for that reason, which was wrong: `review` pays from the run's own ceiling, not from that variable. The server now says whether asking by hand could work at all. Nothing to interpret, no credential and a ledger that cannot cover the ceiling are walls and hide the button; the automatic pass being off is not, and keeps it. **Genesis already had a separate budget**: the envelope of feature 022 lane B, $25 a week, gated before a chat turn and a launch. It knew only about Genesis, it leaked — the embedding path and the engineer sweep reserved without asking — and its name collided with the ledger's own run envelopes. So the envelope is gone and `wb_studio/allowances.py` took its place: a named weekly ceiling per kind of work — Genesis research, run readings, benchmark rounds — inside the one US$300 lab week. A second ledger was the other way to do this and would have authorised $600, since `AUTHORIZED_WEEKLY_MICROUSD` caps each ledger at 300; it would also have split `wb budget reconcile` into two pots to check against the providers' exports. The value a person set as the envelope migrates on first read. Genesis keeps a default of $25 and cannot be uncapped — clearing its field restores the default rather than removing the ceiling, because Genesis is the one that spends on its own. Every other kind starts uncapped and draws on what the week has left. The gate is now at seven call sites, including the two that used to skip it. Also here: a global `taskTitle(r, id)` added to reports.js shadowed app.js's `taskTitle(id)` — these scripts share one scope and reports.js loads second — so the whole run workspace threw and rendered nothing. Renamed to reportTaskTitle. And the Results column headed "Actions" was always the tool-call count; its test said so too. This commit also carries the work of a parallel session in the same tree: features 023 and 024 and the 025 spec. Their code is not reviewed here; the full suite passes at 2359 tests with it in place. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… title from the data Feature 024 User Story 2 — the three defects of it that were still open. The other five were closed by other sessions on the same checkout today and are theirs: resolve_service, the false-completion source, the liveness check, the hero label, and BUCKETS = dict(MODES). **A 95% interval that claimed certainty.** leaderboard.uncertainty clusters by task when there are repetitions, which is right and stays — AI-LABS-DIRECTION.md:132 asks for exactly that, so retries cannot buy confidence. But it estimates from the sample variance of the per-task shares, and when every task scores the same that variance is zero: three tasks that all passed printed "100%, 95% CI 100 to 100". Where the estimator degenerates it now falls back to Wilson over the task count, the conservative binomial answer, still counting tasks rather than attempts. A single task keeps returning no bounds: absent is honest, zero width is a false claim. **A rank from an interval the page did not show.** Standings ranked on measures.pass_rate's Wilson-over-attempts bounds while printing uncertainty's clustered ones. With repetitions those differ, so the order came from numbers the reader could not see. Rank now comes from the printed interval and each row records rank_basis so the two cannot drift apart again unnoticed. This changed a pinned expectation, and the change is the point: two setups that ranked 1 and 2 now both rank 1 with a spread of 2. Two tasks cannot separate a setup that passed both from one that failed both — [0.34, 1.0] against [0.0, 0.66] overlap. The old rank counted two repetitions of two tasks as four independent samples. The review that opened this feature said to rank and display on Wilson over attempts. That was wrong and would have overstated certainty on precisely the repeated runs FR-019 adds. contracts/measures.md records the correction rather than quietly shipping a different thing than it specified. **Two partition rules, and the weaker one was the one people open.** report_data.cohorts keyed a round on task hashes and track; leaderboard.rank_records keyed on those plus judge, assistance, world manifest and workflow contract, and stamped an unpinned cohort provisional. So a round report could pool runs graded by different judges, or on different world revisions, and rank them as one measurement — which PLAN.md §1.1 exists to prevent. leaderboard.evaluation_contract is now the one rule; both surfaces call it; the weaker key is gone. **A figure that asserted its subject onto the data.** The over-time chart was titled "Monarch pass rate by run" and every series drawn in the Monarch colour, whatever the cohort held, and report_data put a setup in the trend only when its name contained "monarch" — so a round comparing anything else had no trend at all. The title is written from the series now; the client resolves each series' own family. 240 tests green across the report, narrative, measures and chart suites; 24 of them new. The spec, plan, tasks and contracts for 024 come with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…es it
`smallest_plan` computed n = ceil(4·p·(1−p)/d²) floored at ten tasks — the size
for comparing two INDEPENDENT proportions. `settle` decides the same hypothesis
with measures.paired -> measures.sign_test, which pairs by task and drops ties.
Below six discordant pairs that test cannot reach p<0.05 at any win count, not
even a clean sweep, and an improvement that flips a fifth of a ten-task set
produces two. So the lab's sizing function recommended, and its floor enforced,
experiments its own settling function could never decide — on sets of ten,
which is every tier set in the repository.
The declared minimum effect is the share of tasks a real difference is expected
to flip, so it is the expected discordant-pair rate and the size follows from
it: n = ceil(6/d).
measures.wins_needed(pairs) fewest wins reaching p<0.05, or None when
no win count can. Derived from sign_test.
measures.minimum_discordant_pairs() six
measures.settleable(...) whether a size can produce a verdict, the
assumed flip rate stated in the sentence,
the sufficient task count named
smallest_plan now carries a `power` block and **refuses rather than trims**.
Capping silently at the population is how an experiment that could never
conclude reached a launch.
Repetitions deliberately do not enter the sizing. They raise confidence in each
task's pass share and never change how many tasks can disagree, which is what
the sign test counts (AI-LABS-DIRECTION.md:132).
docs/rounds/2026-09-11-resized-hypotheses.md re-sizes what was queued. Two
things it found. TRACK.md's "7 hypotheses proposed" counts cards whose kind is
`hypothesis`; only one carries a record, and three of the other six are the
duplicates the watcher loop made on 10 September. And that one record's
population is a filter naming a single task id — one task yields at most one
discordant pair, so it cannot be settled as written at any size or any price.
It is a fair question asked of a population that cannot answer it, and the plan
now says so before money is reserved.
Pinned expectations moved with the rule; the tests carry the arithmetic inline.
132 tests green across the hypotheses, measures and report suites.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e screen still
Stage S0 of the Genesis voice work: pay the accessibility debts before adding
motion, and close the blind spots in the check that is supposed to catch them.
WCAG 2.2.2 Level A: motion that starts by itself, runs past five seconds and sits
beside other content needs a way to pause, stop or hide it. The Studio had five
such animations and no such mechanism anywhere.
ui.css .status.running a pulsing square, for the length of a run
graph.css .bp-wire.live a wire whose dashes flowed forever
graph.css .bp-node.live-running
style.css .node.running the older lane view's ring
style.css .node.running svg an icon rotating every two seconds
None of them carried information. A static mark beside the word "Running" says
the same thing, a dashed blue wire is already legibly live, and the research for
this feature found the pulse actively harmful: it keeps going confidently at a
crashed turn, which turns "I can trust the indicator" into "I must go check".
The turn stream now has a **Pause updates** control, which is 2.2.2's other
clause — content that updates itself in parallel with other content. A turn
re-renders for minutes while the rest of the workspace stays interactive.
Paused, the turn keeps running and keeps being recorded; only the screen holds
still, and the newest state is painted when the reader asks for it.
Four new checks in test_static_csp.py, each of which the existing ones missed:
new WebSocket("wss:// an external resource that does not look like one;
every existing pattern keys on the http(s) scheme
RTCPeerConnection worse — CSP does not govern WebRTC at all, so a
browser would not refuse it either
animation: … infinite the 2.2.2 rule above, enforced
motion opt-in motion must be declared inside
@media (prefers-reduced-motion: no-preference), not
switched off afterwards. A `reduce` override ships
full motion to browsers without the feature, and it
cannot stop a transform written from JavaScript —
which is exactly how an amplitude-driven microphone
indicator would reach a reduced-motion reader at full
swing while the blanket rule at ui.css appeared to
cover it.
Also `[id]{scroll-margin-top}` so an anchored element clears the sticky running
head, and a `:target` mark so arriving somewhere shows where.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…de whether to go Stage S1. Genesis often changes something far from the conversation: it files a card, reads a run, notices the week's research allowance is nearly spent. Until now the only way to say so was a sentence the reader had to act on by hand. **The server never navigates anything.** WCAG's glossary puts a change of viewport on the change-of-context list, and 3.2.5 permits one "only by user request, or a mechanism is available to turn off such changes". A real `<a href="#run/x">` is a user request by definition, and it carries middle-click-to-a-new-tab, copy link, the back button and an address that can be sent to someone else. Moving the screen buys nothing over that and owes a whole conformance mechanism. The follow-me dial, if it is ever wanted, comes later and starts off. `show(route, label, why)` is a plugin tool through the existing seam — two lines in genesis_plugins.MODULES and genesis_schemas.SCHEMAS, no shared file otherwise touched. It validates the route, and the act lands as tool_started / tool_completed in the turn record, the Trace pane and genesis/activity.jsonl. The client renders the receipt as a link plus one line of reason, and writes one sentence into a polite live region declared empty in the markup. The validation is the security half: the route is written by a model, reaches the client and is put in an href. Only known in-app hashes pass, ids may hold only [A-Za-z0-9._@+-] and slash, `..` is refused outright, and 200 characters is the ceiling. 30 tests cover it, including quotes, angle brackets, javascript:, data:, protocol-relative and traversal. Two fixtures in tests/test_genesis_tools.py were repaired rather than worked around, and neither break was mine: - The runs were built with 'hash-<task>' task hashes. Once the FR-012 liveness check landed, every attempt in them read as superseded — correctly — and every measure came back zero. They now carry hashes that match the catalog they are graded against, because a run under test is a live one. - The failure-bucket test expected a `requirement_unmet` id. The buckets are `narrative.MODES` now (FR-013: one taxonomy, derived, instead of a second classifier that disagreed with the narrative on the same attempts), and an attempt that changed nothing is `stopped_short`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stage S2. The live view re-asked for the whole turn every 650 ms — a fresh TCP handshake per tick against the browser's six-connection budget, and a hand-rolled version of the catch-up that the protocol already provides. `GET /api/genesis/turns/<id>/events` is the same shape as the run stream the Studio already serves: Last-Event-ID with an `after=` fallback, one `id:` per frame, a keepalive comment, and a `done` frame carrying the settled turn so the client never has to ask again. Two things that endpoint lacks and this one has. A `retry: 2000` at open, because EventSource has no backoff of its own and would otherwise reconnect on whatever interval the browser picked. And a keepalive relaxed to 15 s, because a turn is minutes of slow tool calls rather than a run's steady tick. The client buffers arrivals and flushes once per animation frame, so a burst of fifty events costs one DOM update rather than fifty. `pollGenesis` is deleted rather than kept as a fallback — EventSource reconnects better than the retry loop it replaces, and two paths to the same screen is how the last three defects in this feature happened. A row that just arrived is marked for 500 ms and fades over 1000. Three deliberate choices in that: - It flashes the DATA landing, never the view re-rendering. Coming back to the tab is not news, and `paintTurn` now knows which rows are actually new. - Not --signal. Red means the current place in this system, and a saturated hue carries the stricter flash threshold under WCAG 2.3.1. - The word "new" sits in the row, so neither colour nor motion is load-bearing, and the transition is declared inside @media (prefers-reduced-motion: no-preference) — the opt-in form the check added in S0 now enforces. ponytail: one thread per open stream, and a held stream owns one of the six connections for its lifetime. One stream per tab; a connection manager if that ever stops being true. Noted in the handler rather than built. 7 tests on the endpoint, covering resume, the after= fallback, the retry line, the done frame, an unknown turn refused before the stream opens, and the no-buffering headers a proxy needs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…not a claim Stage S3. The mark's scale is the microphone's own RMS envelope, so it fails visibly at a muted mic, a wrong input device, or someone too far away. That failure is the point. A timer-driven pulse keeps going confidently at a dead microphone, which in a lab whose founding rule is that nothing grades itself is the same category of error as a fake pass rate. It is a CAPTURE indicator and never an activity one. It appears on key-down, breathes with the signal, and disappears on key-up. The minutes Genesis spends on tool calls keep the step list; one shape may not mean two things, which is the dishonesty the research flagged in the first place. **Nothing reaches the network.** Recognition runs on the device or not at all: Chrome's default ships every utterance to Google with no DPA, no retention statement and no training commitment, and this screen carries provider keys and unreleased benchmark results. Worse, CSP cannot see those requests, no violation event fires, and they do not appear in the network tab — the check would go green while the audio left. So `processLocally: true` is required, `available()` is asked first, and there is no fallback to remote recognition: the button says the browser will not transcribe and the composer still takes typing. A `network` error is reported as "Refused: that would have sent the audio off this machine." Hold-to-talk, not a toggle and not a wake word. Two people share that room and one of them takes calls. It also discharges three requirements at once: the motion never auto-starts so WCAG 2.2.2 is not engaged, there is no silence timeout so 2.2.1 is not either, and there is no pre-roll buffer, so the seconds before the key goes down are never captured. `track.stop()` on release, never `enabled = false`, so the browser and OS recording indicators actually go out. Square, 72px, a filled block inside a hairline gate — `.status:before` grown up and given a live reading, so the fill has full scale to be read against. Scale, never opacity or colour: a flash under WCAG 2.3.1 is a pair of opposing relative-luminance changes, and scaling a solid block on a constant ground changes luminance only where the edge sweeps. Never --signal. The envelope is smoothed (attack 50 ms, release 250 ms) because human syllable rate is 4-7 Hz and an unsmoothed per-frame RMS produces more than three opposing changes a second. Under prefers-reduced-motion the orb is replaced, not frozen: five discrete blocks at 2 Hz. A frozen orb conveys nothing and removes the only live evidence the microphone works. Verified in the browser — the headless one reports reduce, so the fallback is the path that got exercised: `.mic-fill` computes `display: none`, the blocks show, and the word and elapsed counter carry the state in text either way. Amplitude is written with `style.setProperty`, which is CSSOM rather than a `style="` attribute, so the CSP check stays green and `style-src 'self'` is satisfied. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… experiment`
The last of User Story 3. The stratified split, the repetitions field and the
research envelope were finished by other sessions on this checkout today; this
is the held-out gate and a terminal surface over the record.
**FR-022, the once-only rule.** The held-out slate is the lab's one
uncontaminated measurement. A variant that can be retried against it until it
passes is not a confirmation — it is a search over the confirmation set, and the
slate is spent the moment that happens. So a lineage gets exactly one held-out
attempt, whatever that attempt returned: `not supported` is a result, and being
free to retry it is the search this rule exists to stop.
The unit is the lineage, not the record, because otherwise re-wording a claim or
nudging the minimum effect mints a fresh ticket to the same slate. A record's
lineage is its root's id; a fork keeps it. `may_confirm` also requires a
`supported` result on development first: confirming something that was never
promising is not a confirmation.
`check_hypothesis` gains repetitions, id, parent and lineage. Another session
added the slate check while this was in flight, so there were briefly two of
them and two `SLATES` tuples; this keeps one.
**`wb experiment propose` / `confirm`.** No new store — a surface over the record
`genesis_hypotheses` already validates, sizes and settles, which until now was
reachable only through Genesis's own tools, so a person could not check a
proposal without asking the model to do it. `propose` never spends: it sizes
against the paired sign test and refuses before any reservation when that test
could not conclude at the size asked for.
$ wb experiment propose --claim "..." --minimum-effect 0.2 --task-set catalog-50
admitted lineage (new) slate development
30 tasks of 50, 1 repetition, 3 competitors
expected 6 discordant pairs; 6 must be wins to reach p<0.05
maximum $9.15
not launchable yet: the research envelope has $0.00 left this week
**One correction to a refusal another session had pinned.** It advised "raise
repetitions to 3", and that is wrong: the pairing is per task, so
`settleable(10, 3)` reaches the same three discordant pairs as
`settleable(10, 1)`. Repetitions raise confidence in each task's share and never
change how many tasks can disagree. Only the task count moves it. The text now
says so, and the test asserts the equality rather than the old sentence.
213 tests green across the hypotheses, power, slate, corpus and budget suites.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The stratified split, the repetitions field and the research envelope were finished by other sessions on this checkout the same day; the sizing correction, the lineage rule and `wb experiment` were this one. Recorded against the work rather than claimed. Also notes the two collisions with concurrent work and how each was resolved: one duplicated slate check removed, and a refusal that advised raising repetitions corrected — the pairing is per task, so repetitions never change how many tasks can disagree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e machine Stage S4. Off by default and persisted, primed inside the click that turns it on because speak() needs sticky user activation and a later call without it errors on Chrome and fails silently on Safari. No onstart within a second is treated as failure: the text goes to the live region instead of being lost. **Local voices only.** Chrome's `Google …` voices report `localService === false` and are network-backed, so reading an unreleased pass rate through one sends that sentence to Google. One filter closes it, and `utterance.voice` is always set explicitly because the default is not stable across machines. `announce(text)` either speaks or writes to a live region, never both — an app speaking over a screen reader is the most common failure when the two run together. And it never SPEAKS a string that is not already on the page; such a string still reaches the region, because that is text and not audio. That one rule keeps the whole WCAG 1.2 media family out of scope: spoken words that are also written words are an alternative for text, not audio content. It is enforced in the function, not asked for in review. WCAG 1.4.2 wants a way to stop auto-playing audio that is not the system volume: a Stop speaking button, ≥24×24, plus Escape, plus a volume slider on utterance.volume. It speaks the first sentence of the verdict and nothing else. Never the tool events — an agent should not sound like it is being interrupted by its own logs — and never the whole answer, which is on screen already and would be a lecture. A fixed "Genesis: " prefix and a distinct rate, because on macOS speechSynthesis and VoiceOver draw on the same system voices and are otherwise indistinguishable by ear. Verified in the browser rather than reasoned about, by stubbing speechSynthesis.speak and reading what would have been said: priming utterance is silent yes text on the page is spoken yes text NOT on the page is never spoken yes (it reaches the region instead) every spoken string carries the prefix yes nothing is announced twice 1 utterance, not 2 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stage S5, and the checkbox IS the conformance mechanism. WCAG 3.2.5 allows a
change of context when it is "initiated only by user request, OR a mechanism is
available to turn off such changes" — that `or` is the whole design budget, and
it is one checkbox. S1 shipped the link first on purpose; this is the opt-in on
top, and the link never goes away.
Four rules, none of them a disclaimer:
- **Off by default, per browser, in localStorage.** Never a server setting, or
one person's toggle moves the other's screen.
- **Break on any gesture, for the rest of the turn.** One `{once: true}` listener
each on wheel, keydown, pointerdown and touchstart. Google Slides is the
precedent: following stops the instant you act, no confirmation, no ceremony.
Escape releases it too.
- **Scroll moves, focus never does.** Focus is on the change-of-context list as
well, and taking it mid-keystroke breaks actual typing. It moves only when the
person activated the link themselves.
- **Never follow when it cannot be watched.** A hidden tab, or a surface the
reader is not on, is a change of context with no witness: it degrades to the
link silently.
The bar is persistent, not a toast — Miro had to retrofit an explicit stop
because people got stuck in a follow that outlived the session, and a toast that
carries the only way out is a way out that vanishes. After a break it offers
Resume rather than pretending nothing happened.
Also extracts one router. The route switch lived inline inside the popstate
listener, so "go to this address" existed only as a side effect of the browser
going back, and anything else that wanted to route had to write a second copy.
`window.goRoute(hash)` now serves both.
Verified in the browser through the real arming path:
armed while a turn streams "Following Genesis"
a wheel event "Following paused — you took the wheel"
the button "Resume following", and it does
nothing armed, a wheel event no effect, correctly
Stop following dial off, bar gone, hash untouched
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… will not Stage S6, the last of the voice work. It exists only because S3 refuses the easy path: Chrome's SpeechRecognition without processLocally ships every utterance to Google under no DPA, no retention statement and no training commitment, and the request is invisible — CSP cannot see it, no violation event fires, nothing appears in the network tab. So S3 does on-device or nothing, and this is the fallback for a browser that cannot. What changes is whose terms apply. The clip goes to the Studio's own origin, so `connect-src 'self'` is untouched and there is no API key and no ephemeral token in the page. The Studio calls the provider with the key it already holds, under API terms that do not train on submitted audio. **Reserved before it is sent, settled from what it cost**, like every other provider call — an unmetered path to a paid API is how a weekly ceiling stops being one. It also asks the research allowance first, so voice cannot quietly eat the money a round needs. The ledger behaviour is the part worth reading: refused before sending nothing reserved at all service refused it settled at zero; nothing was billed no answer came back the hold STANDS — unknown billing is not zero settled per second of audio, never the ceiling Deliberately narrow: one clip, one call, 4 MB and 120 seconds, five audio types, and the buffer is dropped the moment the text returns. Nothing is stored. The transcript lands in the composer as an editable draft and is never auto-sent, which is also the text-parity anchor for WCAG 1.2.1. Two bugs found by running it rather than reading it: - The route first sat inside the JSON branch of do_POST, which reads and parses the body before dispatching — so audio bytes were JSON-parsed and every request died on "Expecting value: line 1 column 1". It now sits before that read, after the auth and origin checks. - `up()` nulled `recorder` before testing it, so the status line the recorder sets was cleared immediately; and the media tracks were stopped before `recorder.stop()`, which loses the final chunk. Verified against the running fixture: a non-audio type, an over-long clip and a host with no key each come back 400 with the sentence a person would want to read. 15 tests on the module. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The orb became defensible the moment there was real audio: amplitude-driven motion is a measurement, and it fails visibly at a muted microphone. It stays a capture indicator and never an activity one. Records the finding that shaped every stage — the CSP test cannot see the privacy risk, because browser speech recognition leaves over a channel that raises no violation event and shows nothing in the network tab — and the eight things refused with the reason for each. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ered in The voice and presence work was asked for mid-implementation, so it is appended rather than renumbered. Records why the indicator is defensible now that it measures a real signal, and the finding the CSP test structurally cannot make: browser speech recognition leaves through the browser's own machinery, raising no violation event and showing nothing in the network tab. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… not define 954676a staged wb_studio/app.py whole and swept in four lines another session had mid-edit on this shared checkout. One of them imports default_ledger_path from wb_orchestrator.budget, whose definition stayed uncommitted — so `import wb_studio.app` raises ImportError on a clean checkout of that commit. The full suite stayed green throughout because it runs the working tree, where the function exists; only a fresh checkout sees it. This is the function, unchanged, by its author. One rule in one place for where the shared weekly ledger lives: a caller that guesses a different path reads a different week's spend and lets through what the ledger would have refused. Found by ailabs-9c, whose tree it came from. Repaired rather than amended: 954676a is already the base for other work on this branch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ects it found US4, the fitness function. Configure once, execute many: a workflow is built one time and run N times, a harness pays full price every run, and the crossing point between those two lines is Monarch's whole value proposition. That curve needs the split, so this makes the split real and trustworthy. Most of it was already recorded and nobody was reading it. The arm has written per-phase cost and wall clock since feature 002 — monarch.py:463 for cost, and the clocks in each phase's `finally` so a timed-out attempt still says where the deadline passed. The plan's premise that a new `authoring_ended_at` timestamp was needed is wrong; the clocks are there. What was missing was a reader. The blocker, which no task in the plan covers: the Studio's result row dropped `phases` entirely (app.py:790), keeping one flat cost and one duration. That dict is the only input `measures` ever gets, so the split was out of reach of every report. Four lines. Two live defects found on the way, each with a test that reproduces it first: `attempt_seconds` summed `run` — the whole attempt — together with `authoring` and `execution`, which are its parts. Every Monarch row reported roughly twice its true duration: 200 seconds for a 100-second attempt. The function thirty lines below already skips `run` and `model:*` by name, so the module knew the rule; this one did not follow it. It is also the number the "faster than a harness" claim is read off. No fixture carried `run` beside real phases, so nothing caught it. `_phase_block` read an unpriced phase as `cost_usd or 0.0`. `PhaseMetrics.cost_usd` is already None when nobody could price it, so the discipline was correct at the arm and lost at the reader. An attempt whose cost cannot be read holds its whole ceiling against the week instead of settling — up to US$ 25.00 apiece — so rendering it as US$ 0.00 tells a reader the round was cheap while the ledger still holds the money. New in measures, all pure functions over stored rows: - `phase_cost` keeps three states apart. `n/a` is a competitor with no such phase — a bare model never configures anything, and calling that zero makes it look free at the one thing Monarch charges for. `unknown` is a phase that ran unpriced. - `cost_by_phase` reconciles the parts against the total and returns anything the named phases do not claim as `unattributed` rather than dropping it. Monarch only pre-creates `authoring` and `execution`, so discovery spend really does fall outside them; `reconciles` is how a reader learns the bucket was needed. - `time_by_phase` reads the clocks already recorded and checks each attempt's parts against its own total, one second of tolerance. - `cost_per_pass` returns NO_PASSES, not unknown, when nothing passed: the record is complete and the answer is undefined. A reader told "unknown" goes looking for data that does not exist. Staged by filtered patch, not by file: this checkout has other sessions editing the same test files, and two of their hunks were excluded by name. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nd, a drift that named nothing The scheduler wrapped its "are the dials on?" check in `except Exception: pass` and then fell through to running the day's jobs — six of which reach paid model turns. It was reachable: `Autonomy.read()` raised KeyError on any dial value this version does not know, which an older file or a hand edit produces. So settings nobody could read unpaused the research loop. Both ends: an unrecognised value falls back to off, and settings that cannot be read stop the jobs the way Pause already does. A gate that fails open because it crashed is worse than no gate, because it reads as one. The episode shim read whatever Content-Length claimed. The competitor under test is what calls it, and `wb_arms.monarch` binds it to 0.0.0.0. Capped at the 8 MB the Studio's front door already allows, so a body the door accepts this accepts; both the tool path and the REST path answer 413. ConfigDrift refused to resume without naming the setting that moved or the file it lives in. The run row already stores the configuration, so the message now ends "Changed since the run was recorded: plan.repetitions". `code_identity` keeps a note saying why it is not cached: it answers whether the tree is the same on both sides now, and a cached answer lets a coordinator admit a worker running code it is not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ne value Six definitions were referenced nowhere — not by code, tests, the Studio's JavaScript, or the docs. Checked each across the whole repository before removing it. `legacy/adapter.py` goes whole: `run_legacy` was its only entry, `ensure_venv` and `legacy_python` existed only to serve it, and it hardcoded a clone path on a machine directory that no longer exists. `legacy/importer.py`, which the CLI and two tests do use, is untouched. `enterprise_deploy` loses 137 lines — `start_candidate`, `configure_services`, `candidate_compose` and the AWS adapter script — 44% of the file, unreachable from its own `main()` and never tested. It reads as a deployment path someone stopped halfway; the git history keeps it. `_cost_section` opened with `dollars = True` and branched on it, so the else arm and the `base` it needed were unreachable and both arms computed a `guide` nobody read. It is two lines now. `_round_source_line`, `_mtable` and `chance_sentence` were each superseded by code that shipped. Also eight imports and four locals that nothing used. Pyflakes over the whole backend goes from 33 findings to zero. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…efused US4 finished. `per_execution` reads a run-only round as the engine alone — cost and wall clock per execution, per task — which is the second half of the break-even curve: a workflow configured once and run N times against a harness paying full price every time. `configure_usd` there is NOT_APPLICABLE, never zero. The recipe was authored once by `wb monarch recipes`, outside the round; a zero would say the engine is free to set up, which is the opposite of what the curve exists to measure. A task nobody could price stays unknown on its own row and blinds only the cohort figure, which cannot be summed without it. T093, the paid recipe round, is refused twice over and both refusals are right: $ wb monarch recipes --tasks tasks/tier-simple paid launch refused: Monarch instance not verified: milestone M5 exit 2 The refusal fires before any client is built, so nothing was requested, reserved or spent. And `tasks/dev-50` does not exist — `wb corpus split` has never been run, so there is no development slate to author recipes for. Drawing the split is free and is the first step; the paid launch then needs M5 and a named human approver per decision D5. An agent never approves its own round. The round sheet records the money fact that decides whether it is affordable: an attempt whose cost cannot be read holds its whole ceiling against the week rather than settling, at US$ 25.00 apiece, so ten unreadable attempts hold the entire US$ 300 week. A silent Langfuse outage does not fail the round, it exhausts the budget. tasks.md records three premises of the plan that were false — the phase cost and the phase clocks have existed since feature 002, and `n/a` belongs in the reader rather than in api_loop, which never writes phases at all. Staged by filtered patch again; one hunk belonging to another session excluded. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…y calls Recovery, not new work. Two sessions built most of User Stories 1 to 3 on this shared checkout, committed the CLI wiring, and ended before committing the implementations. `specs/024-architecture-search/tasks.md` records all of it as complete. It was not: one `git checkout` and three user stories were gone. HEAD was broken in the same way $3d87f47 repaired. HEAD's cli.py routes `wb corpus split` to `slate.split` and `wb budget envelope` to `genesis_access.Envelope`, and HEAD defines neither. The full suite never saw it because it runs the working tree, where both exist. git show HEAD:wb_orchestrator/cli.py | grep -c 'corpus_cmd == "split"' -> 1 git show HEAD:wb_orchestrator/slate.py | grep -c '^def split' -> 0 What lands, by user story: US1, the split (T060-T065). `slate.split` draws a development and a held-out slate stratified by difficulty tier and domain, freezes each with a manifest and a hash, and refuses a redraw. `_write_split` writes everything or nothing: it used to write the held-out slate file by file, and the FR-018 refusal triggers on any *.json in that folder, so a failed copy retired the name permanently. `corpus.structural_difficulty` is the objective measure it draws on, counting seeded services rather than keys — under the repaired world every scored task lists all 48 apps' empty defaults, so counting keys would score every task 48 and say nothing. US2, the instrument (T045-T059). The second failure classifier is gone: `failure_analysis.BUCKETS` is now `dict(narrative.MODES)`, one taxonomy rather than two disagreeing ones. The third inlined Wilson in difficulty.py is replaced by `measures.wilson`. Cohorts carry their provisional note. US3, the envelope (T075-T079). `Envelope` is the standing weekly research allowance FR-035 asks for. `_spent` looked for the ledger only beside the genesis folder, which is the fixture layout; on a real Studio it found nothing, read nothing spent, and reported the whole envelope available — $50.00 available with $40 held. Both callers are gates (`background_wanted`, `may_launch`), so the allowance could not fire at all. It now uses `default_ledger_path` and fails closed. Two of these repairs are ailabs-9c's, sitting inside blocks too large to separate by hand: the `_write_split` all-or-nothing write and the envelope's ledger lookup. Both are the kind that only appear outside the fixture layout, which is why they survived a green suite. Staged file by file from a filtered patch, and test_budget.py split mid-hunk at `test_a_released_hold_stops_blocking_the_weeks_after_it` — the 102 lines after it are ailabs-a1's live work on the ledger release and stay uncommitted. budget.py, cli.py and reconcile.py are excluded for the same reason. Full suite green before staging: 2539 passed, 3 skipped. The staged tree was verified on its own, not on the working tree: extracted from `git write-tree`, imported, and compileall clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ce strip broke US5. The round now opens with the argument it is for: accuracy with its uncertainty, then what it costs to configure once and run again. The curve. A workflow is built one time and executed N times; a harness pays full price every run. Where those lines cross is Monarch's whole value proposition, and nothing in the evaluation landscape plots it, because no other subject has a reusable artefact. Costs are per successful task (FR-028), so a competitor that is cheap because it fails is not cheap. Three outcomes kept deliberately apart. A crossing is a measurement. `none-in-range` means the lines have not met inside the evidence and the figure says so in words instead of drawing a projection. `never` means one execution costs at least as much as one request, so no n can ever cross — presenting that as "not yet" would be a claim the data refutes. A round with no competitor that configures anything gets the sentence, not an empty chart; a bare model pays per request by construction, and giving it a configure step would invent the very asymmetry the figure exists to measure. The gap list has no flat `items` key, on purpose. A `code-reading` is somebody reading Monarch's source and a `confirmed-result` was measured; listing them together invites a reader to weigh them the same, so a caller cannot reach for the obvious field and render them as peers. Three audiences change the wording and nothing else — same page, same setups, same numbers — and an export drops code readings and says how many went, because a silent omission reads as "there were none". T105 is built somewhere other than the plan says, for two reasons. `wb report` goes through `wb_report/report.py`, which never imports `report_data`, so the gap list does not exist on that path and threading `--audience` there would produce a second `sortable` — a parameter carried through signatures no body reads. And report_data.py:38 records Lucas's ruling of 11 September: one report, every reader sees the same page, the audiences gate removed. So the audience is a query parameter on the Studio's report route, scoped in the code to the gap list's wording, and an unknown value falls back rather than erroring. `proposal` is what a held-out confirmation produces: a written specification, not an executable artefact. Only a `supported` outcome on the held-out slate qualifies — development is where the search happens and is contaminated by construction, since every variant that reached it was chosen because earlier ones did well there. The browser suite caught a regression from stage S4: the voice strip did not wrap, and at 375px the hold-to-talk button, the state text and the volume slider together overran the composer and pushed the whole page sideways by 54. Verified fixed in the pane at 375px, overflow 0. Browser suite 31 pass, 0 fail — two earlier failures were port contention with another session's suite, not defects; the editor path was driven by hand to confirm. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
T109 to T111. Feature 004 was listed as "specified, not built" when wb_orchestrator/monarch_recipes.py is complete and the CLI refuses it only on the M5 gate. The file map gains the two slates, the research envelope, the front-door secret and the fitness measures. Decision D10 is recorded as satisfied, with the two defects found in satisfying it: money no named phase claims is reported as unattributed rather than dropped, and an unpriced phase stays unknown because it holds its whole ceiling against the week. T112 is NOT done and is not marked done. Graphify is not installed here — no skill, no binary — and graphify-out/ does not exist at all. CLAUDE.md tells every agent working in this repository to read graphify-out/GRAPH_REPORT.md before answering an architecture question and to refresh it after code changes, and that file has never been there. So either every agent has silently skipped the instruction, or some have claimed a refresh they could not have performed, which is the exact thing the constitution's wording was written to prevent. Either install Graphify and build the graph, or strike the instruction. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… broken T107. 2542 passed, 3 skipped, 0 failed against the T001 baseline of 2364. The count moved by more than this feature added: three sessions committed to this checkout through the day, one deleting about 1,300 lines of dead code and its tests. ruff --select F811 is clean across every file this feature touched. A peer found three shadowed definitions elsewhere the same day, each of which looked like working code and none of which ran — the same shape as this feature's own central defect, a phase block dropped between the arm and the reader. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…arch needs The one remaining blocker that was mine to clear. The task list named `tasks/dev-50`, a slate that had never been drawn — and `wb corpus split` is free, so it is drawn rather than left as a note. development 50 tasks simple 17 medium 17 complex 16 held-out 50 tasks simple 17 medium 17 complex 16 frozen: tasks/development-50 (sha e724…), tasks/held-out-50 (sha 27b0…) Stratified by difficulty tier and domain over 800 usable tasks across seven domains, seed 20260911, every drawn task's score, tier, domain and hash recorded in tasks/split-manifest.yaml. The slates share no task, and a redraw of the held-out slate is refused rather than performed. This is the split `AI-LABS-DIRECTION.md` asks for in three places and that no implementation existed for. Without it every win the search reports is unfalsifiable: search against a grader finds whatever the grader rewards, and this lab's grader has already been wrong once, badly enough to void four rounds. Recipes must be authored against the development slate only. Authoring a recipe means reading a task closely enough to build a correct workflow for it, which is precisely the contamination the split exists to prevent. The held-out slate is run once per lineage, with recipes it had no part in shaping. T093 still refuses on its first block, which is the one that matters: `wb monarch recipes` exits 2 on the milestone M5 gate before a client is built, and a paid launch needs a named human approver. Neither is mine to clear, and an agent never approves its own round. T112 checked exhaustively rather than assumed: no PyPI distribution, no entry in any installed marketplace, no plugin, no binary, no skill, no graphify-out/ directory. The constitution forbids claiming a refresh that did not happen. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…t since Closes the T112 finding with its cause. HANDOFF-2026-09-02.md:49 lists "Initialize Graphify on the repo (/graphify-init)" as a to-do, and it was never done. CLAUDE.md nonetheless opens its Graphify section with "This project has a graphify knowledge graph at graphify-out/" and tells every agent to read GRAPH_REPORT.md before answering an architecture question. So this is not a missing tool. It is an instruction that has been wrong for nine days, and every agent working here has either silently skipped it or claimed a refresh it could not have performed. Two ways to close it, both a person's call: run /graphify-init, or delete the section. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Seven places in this file described `graphify-out/` as an existing asset: the stack summary, the mental model, stages 0 and 6 of the pipeline, the working rule, the file map, and a whole section opening "This project has a graphify knowledge graph at graphify-out/". The directory has never existed. HANDOFF-2026-09-02.md:49 listed "Initialize Graphify on the repo (/graphify-init)" as a to-do. It was never carried out, and the instruction has been telling every agent since to read GRAPH_REPORT.md before answering an architecture question — so each has either silently skipped it or claimed a refresh it could not have performed, which is what the constitution's own wording was written to prevent. Graphify is also not obtainable on this machine: no distribution, no plugin, no binary, no skill, no `/graphify-init` command. Corrected rather than deleted. The intent survives and the rules are kept for the day it is built; what changes is that the file now says the graph is not there and points at what to read instead — the file map, HOW-WORKFLOWBENCH-WORKS.md and STATE-OF-THE-PROGRAM.md. The rule about honesty survives outright, because it is about the claim and not about the tool: never claim a refresh that did not happen. This does not close feature 024's T112, which asks for a refresh. It removes the reason T112 was dangerous. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…o whoever runs it The constitution requires an attempt count and a cost band before any paid launch. Both are computable from the code and the slate that now exists, so the round sheet carries them: 50 tasks, up to 3 authoring attempts each, 150 at most, US$ 10.50 to US$ 33.00 against a US$ 300 week. Stated beside it, because the band is the smaller number and the misleading one: the band does not bound the reservation. Each attempt reserves its US$ 25.00 ceiling before dispatch and settles from the receipt, so an attempt whose cost cannot be read holds the whole ceiling rather than the cents it spent. Ten of those hold US$ 250 of the week. Confirm Langfuse is answering first. T093 still needs milestone M5 and a named human approver, and an agent never approves its own round. What is removed is the arithmetic between the approver and the decision. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Records what the feature actually taught, and links the CSS-specific check another session wrote up at AI-LABS-STUDIO-DESIGN-SYSTEM-2026-09-09.md "Removing a rule (11 Sep)". Every defect here reads the same way: something present that looks absent, or absent that looks present. A phase block written for nine days and consumed by nothing. A total summed with its own parts. A held reservation rendered as free. HEAD importing a symbol no commit defined, three times, each with a green suite. A pipeline stage pointing at a knowledge graph that never existed and never once erroring. A class built by string concatenation reading as dead, and three shadowed definitions reading as live. The general form: verify the consumer, not just the producer. A value written and never read is indistinguishable from one never written until someone checks the far end. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… tree My own commit had the shape it was describing. 02da17b cites AI-LABS-STUDIO-DESIGN-SYSTEM-2026-09-09.md "Removing a rule (11 Sep)"; the section was real in the shared working tree and in no commit, so on a clean checkout the citation pointed at nothing. git show HEAD:docs/AI-LABS-...md | grep -c "Removing a rule" -> 0 grep -c "Removing a rule" docs/AI-LABS-...md -> 1 I wrote the citation and it resolved when I looked, because I looked at the working tree. The consumer is `git show HEAD`, which is the check I had spent the day telling other sessions to run. Caught by ailabs-07, whose section this is; the file's only diff is theirs, so staging it whole is safe here. What it records is the CSS half of the same lesson: a class built as 'view-' + name reads as dead to a text search, and an id selector survives a class-only pass — #leaderboard-panel outlived its panel in two sheets because a mixed list keeps the rule alive on its live half. It says regenerate the prefix list rather than trust a written one, including the one it prints, after its author's own list came back two prefixes short immediately after publishing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… the QA pass over both Several sessions' work landing together. This is not one author's change set, and the message does not claim it is: the external-world work, the Genesis mission, partner-model, report-cycle and reasoning-evidence work, and the review fixes below were made by different sessions against this shared checkout. It is committed whole because splitting it by file would attribute each file's mixed hunks to whoever happened to stage it, and splitting by hunk is a person's job, not a guess. Feature 026 -- products under test we did not write. AppWorld, tau2 and EnterpriseOps-Gym run out of process behind the same front door: a world is an HTTP endpoint plus an interface document. `wb_world/adapter.py` states the contract read from what the orchestrator already demanded of an episode, so the AutomationBench world satisfies it unchanged. The verdict keeps two halves -- the source's own checker for "the expected result is present", ours for "and nothing else changed" -- and a source's own side-effect finding is recorded beside ours, never merged into it. None of the three is a dependency: AppWorld pins pydantic<2.0 and this project needs >=2.0, so they could not share an environment even if we wanted one. Features 025, 027, 029 -- Genesis builds an architecture a step at a time and one save commits it; owns report analysis, authorship and review as separately recorded turns; and can run research missions. Plus live workflow metrics (028). Fixes found by the review pass over this change set: - grader/grade.py reported `invariant_declared` true for an external task whose `approval_rule_reviewed` was falsy. tau2's importer writes the empty list whether or not a person read a rule, so unreviewed rows claimed a review nobody made. The strict rule is unchanged and deliberate; only the record's claim about it moved. - wb_arms/http_shim.py refused administrative operations by comparing `rest_url`, which carries the service segment, against service-relative paths -- so `/gym-itsm-mcp/api/sql-runner` never matched `/api/sql-runner` and the door was shut in name only. The adapter's own guard was the thing actually refusing them. FR-034 wanted the door not to depend on each world remembering. Now it does not, and a test proves it with a world that guards nothing. - wb_stats/stats.py counted ungraded pairs as infra-dropped, and the report printed them that way. A round whose grading broke read as a round whose machine did. `dropped_ungraded` is now separate, through metrics, report and html. - wb_studio/genesis_reports.py `published()` raised TypeError from Path(None). It is a read, so it now answers like one; `round_report` works over stored job records with no directory. - wb_studio/genesis.py `_dispatch` raised FileNotFoundError on a card whose parent record no longer exists. Historical hypothesis links predate the mission store. - wb_studio/genesis_harness.py settles a never-claimed reservation at $0 in the `finally`. A turn stopped between `ledger.reserve` and `ledger.claim` left the hold unclaimed for ever, since `finish_run` releases only unallocated capacity and only a named person may release an unclaimed hold. A dispatched request still settles None/'error': its billing is unknown, not zero. - wb_studio/genesis_config.py `effort_for(step, route)` drops a thinking level the named route's API cannot express. The partner default 'medium' was being demanded of chat-completions routes, which carry no reasoning-effort parameter, so `genesis.chat` raised before reserving and no turn ran on those routes at all. An effort a person names for one request still raises. - wb_studio/narrative.py drops `elif not actions: unclassified`, on Lucas's decision: without a trajectory the checker's own facts still name the mode. The comment at narrative.py:181 records why it must not return. .gitignore keeps 66 MB of feature-026 verification scratch out of a public repository. The round sheet in monarch-benchmark/docs/rounds/ is the record; the four small JSON files it cites and the eog report stay. No AppWorld content is committed -- its licence permits public redistribution only in packed form. QA: 2949 passed, 9 skipped. The one red, test_budget.py::test_concurrent_processes_cannot_overbook, died with BrokenProcessPool and "OpenBLAS error: Memory allocation still failed after 10 retries" under a concurrent review fleet; it passes alone in 4.14s. HEAD and the working tree both carry no dangling local imports across 352 files. Open, deliberately not patched: `route_for` resolves the chat and reading steps to the partner model exactly, with no fallback, while every other step degrades to strongest or cheapest. That is documented intent -- a lab must say which model spoke -- but it means one provider outage silences Genesis's conversation and reading while review, patch and critic keep working. Whether that trade is wanted is Lucas's call. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…used it
`railway up --no-gitignore` uploads everything .railwayignore does not name, and
feature 026 brought heavy working directories into the repo that it did not name:
.external at 1.6 GB (the external benchmarks' own environments), .venv at 405 MB,
.tmp at 331 MB, and the round by-products under out/. The upload reached
411,918,174 bytes and returned 413 Payload Too Large.
None of it belongs in a build context. The Dockerfile copies source and runs
`uv sync` to build its own environment, so a local .venv is dead weight; the
external benchmarks run out of process and are never in the image at all.
out/studio stays, because the entrypoint seeds the volume from it on first boot --
but out/studio/genesis does not. That is 77 MB of local Genesis turns and cards that
accumulated under the seed folder since hosting began. The entrypoint seeds only when
the volume's studio directory is empty, and production's volume holds 2.1 GB, so it
shipped on every deploy and was never read. The pre-hosting runs beside it still seed
a fresh volume.
Deployed and verified: image built and pushed, container booted ('AI Labs Studio:
http://0.0.0.0:8765'), and the public host answers 401 through the basic-auth gate.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ied before it was believed A fan-out over the change set just committed: sixteen reviewers by subsystem, every finding then handed to two independent skeptics prompted to refute it and defaulting to refuted when uncertain. 54 findings survived; these are the six in this half that were small and unambiguous enough to fix without reopening a design decision. The rest are in the handoff. - http_shim.py refused administrative operations on the REST surface only. `POST /fetch` takes a URL the competitor wrote and handed it straight to the world, so the shorter way in was never closed -- and the guard's own comment claimed the door did not depend on each adapter remembering. Both surfaces now ask one helper, which strips a known service segment first, the way an adapter's router does. Three URL shapes and an ordinary call are pinned. - orchestrator.py read `self._budget_run_id` bare on the external attempt path while `_admit` only sets it when the round has paid liability left to reserve. A resume taken after every paid competitor is complete, with a scripted control still queued, raised AttributeError inside the attempt -- which the blanket handler filed as a scored `agent_error` against the competitor. Now `getattr`, matching the guard `_execute`'s own finally already used. Latent today: no shipped plan pairs a scripted competitor with an external product. - measures.py `break_even` answered for UNKNOWN and NOT_APPLICABLE but not NO_PASSES, the third sentinel `curve` hands it whenever a side never passed a task. It reached float() and took the whole round report down with a TypeError. A cohort that never passed is not an unknown -- we know exactly why there is no number -- so it returns its own reason. The test that should have caught this is named `..._unknown_and_no_passes_...` and never checked no_passes; it does now. - report_data.py `paired_table` and `matrix_cells` still filtered on infrastructure alone after `ungraded` was propagated into `measures.evaluated`, `pass_rate` and `leaderboard.task_shares`. An attempt our own checker could not grade was counted as a verdict in two published tables. - evidence.py `_plain` stripped the four characters of `\?\` without turning the remaining `UNC\` back into `\`, so the two forms of one UNC path stopped comparing equal and `write_attempt` rejected its own journal on a network `--out`. - tools_openapi.py `build_tool_spec` raised IndexError on a tool whose description is present but only whitespace: strip leaves no lines and [0] took the service's whole interface document with it. Also restores http_shim's `__main__`, which referenced `Episode` after feature 026 deleted the top-level import and so raised NameError immediately -- the documented way to serve one task's world by hand for the live checklist. Only files this session changed are staged; several other sessions are working in this checkout and their edits are left where they are. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…runs Thirteen files conflicted; every resolution keeps both sides' behaviour. The three that were not a union: - `SchemaInterfaces.rest_url` was a copy of `world_url` made when feature 026 put a world behind an interface. The copy had already lost the tenant substitution, and main's Gmail path prefix (772b45d) landed in `world_url` where the shim no longer looked. It delegates now, so there is one implementation to fix next time. - `config_repository.safe_path` knew two top-level config files. Feature 026 added three more side-effect lists beside them, so every snapshot of the current tree was refused as an unsupported path - 30 of the 41 failures. - `resolve()` keeps main's `runtime_root` and `source` but not `audiences`, which nothing passes: the audience gate went on 11 September and the `audience` that remains is the gap list's wording. Two test doubles went stale against main's code rather than the other way round: the CLI banner now reads `config_source`, and the read-only reporting guard named `wb_studio.reports.Store`, a name main's refactor removed. The guard is patched at `wb_results.store.Store` now, which is what it meant. Verified on the world revision CI pins (upstream 1.0.6 at 4a8e106, not the suspended evalrepair fork the working copy vendors): 3179 passed, including the two Monarch files that deadlock in a shared process. The two remaining failures reproduce on faeac3d before this merge and are not from it: `test_display_names` (a setup named only in `models` is dropped when `arms` is set) and `test_studio_skills_and_debrief`.
…st's own route Two CI failures, two unrelated roots. setup_names resolved ids through `setup_ids`, which picks arms OR models and never both. A job carrying an arm list therefore left every id in `models` unnamed, and the report printed the raw internal token — the one thing that function exists to prevent. Naming is not selection: anything that could reach a page gets a name, from either list. `setup_ids` itself is unchanged, so which setups a report *shows* is untouched. test_work_now_respects_the_pause_and_the_queue inherited the global partner default for its route. `reading` resolves the partner exactly and never substitutes, so the test's outcome moved when the lab changed which model it picks — a test about the pause and the queue failing over an unrelated config default. Pinned to its own stub route, the way tests/test_studio_watcher.py already does it. 192 passed across the report, naming, watcher and skills suites. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
53 commits: features 024 to 029 (the search loop, Genesis voice and live build,
the engineer and critic loops, external benchmark products, research missions),
then a merge bringing in
main's versioned benchmark configuration andrecoverable Studio runs.
The merge
Thirteen files conflicted. Most resolutions keep both sides. Three did not:
SchemaInterfaces.rest_urlwas a copy ofworld_url, made when feature 026 putthe world behind an interface. The copy had already lost the tenant substitution,
and the Gmail path prefix from
772b45dlanded inworld_urlwhere the frontdoor no longer looked. It delegates now, so there is one implementation.
config_repository.safe_pathallowed two top-level configuration files. Feature026 added three more side-effect lists beside them, so every snapshot of the
current tree was refused as an unsupported path.
resolve()keepsruntime_rootandsourcebut notaudiences, which nothingpasses. The audience gate went on 11 September; the
audiencethat remains isthe gap list's wording.
Two test doubles had gone stale against the code rather than the other way round:
the CLI banner reads
config_source, and the read-only reporting guard namedwb_studio.reports.Store, a name the report refactor removed. That guard ispatched at
wb_results.store.Storenow, which is what it was for.Verification
3179 tests pass, including
test_monarch_arm.pyandtest_monarch_recipes.pyrunin their own processes, since their server double deadlocks in a shared one.
Verified against the AutomationBench revision this workflow pins (upstream 1.0.6 at
4a8e106), not the1.0.6+evalrepair.10fork the working copy vendors. Thatdifference is worth knowing: 37 of the 41 failures seen on the first run were tests
comparing a frozen run's world revision against the installed one, and all of them
went away once the pinned revision was in place.
Two failures are not from this merge and reproduce on
faeac3dbefore it:test_display_names(a setup named only inmodelsis dropped whenarmsis set)and
test_studio_skills_and_debrief. Fixes for both exist uncommitted in the sharedworking copy and are not included here.
Not in this change
Nothing is deployed. Production serves an image built before this merge.