Status: active roadmap Last updated: 2026-09-13
DESIGN.md says what Sideeye is. This document says in what order it becomes real, what each milestone must prove, and what v1.0 promises. Milestones are ordered by risk: the assumption most likely to kill the project is always the next one tested.
v1.0 is a contract freeze, not a feature count. What freezes at 1.0:
- the Define contract (
sideeye.tomlformat and the L0/L1/L2 levels), - the report schema (JSON),
- the exit codes (0 PASS / 1 FAIL / 2 UNKNOWN / 3 SETUP ERROR),
- replay compatibility for saved cases,
- the MCP surface (tool names, input schemas, the isError rule — added by decision in the freeze audit, 2026-08-13).
Before the v1.0 tag, any release could break any of these. Since the tag, a change docs/contract-freeze.md — the normative declaration of all five surfaces — does not allow is a breaking change whichever version carries it: the version number does not record it, each is ruled on its own and licenses no other, and that page records each one (ADR 0064).
Goal: prove the engine on the easiest ground. This milestone is the feasibility spike.
Scope:
- Linux only, single operation, dynamically linked targets.
- Crash-point exploration at file-operation boundaries; deterministic, logically addressed crash points.
- L0 built-in atomicity invariant (state-dir snapshot comparison). No checker support yet.
- State snapshot/restore per world (DESIGN §14-11).
- JSON + text reports with exploration counts; the full exit-code contract.
- Reproduction by re-running the operation with the crash point named in the report, printed as a
reproduceline the caller can paste.sideeye replay <case>moves to v0.2, with the case storage it needs: this milestone listed the command while listing storage under v0.2, and a case that was never stored cannot be replayed.
Acceptance:
- A toy target with a planted delete-before-rename bug yields FAIL with a logical crash point, reproduced 10/10.
- The corrected toy target yields PASS with exploration counts.
- UNKNOWN and SETUP ERROR are demonstrably reachable — each verdict path falsified once. A gate whose failure paths were never seen firing is not a gate.
- The printed
reproduceline is executed by the suite, not read. It was wrong twice while looking right.
Risk retired: interposition works at all; crash points are deterministic and reproducible.
Delivered beyond this scope, because it cost less than deferring it: macOS interposition and OS parity (originally a milestone of its own, below), and L2 checkers with falsification (originally part of the Define-contract milestone). Those milestones keep their remaining scope.
Goal (as shipped): stop refusing every target that creates a process, without ever guessing about one.
This milestone was not on the original roadmap — the slot was promised to the Define contract, which moves to v0.3 with its scope unchanged. What forced the queue-jump: pointing v0.1.0 at its first real target (the v0.4 dogfood subject) ended at child_process_detected for the shape every shim, wrapper and launcher shares, and the attempt surfaced a v0.1.0 defect in which observing a vfork killed the target. Roadmaps yield to measurements; that is what they are for.
Shipped:
- Containment: every target runs in its own process group, killed and reaped as one, so nothing a target starts outlives the engine's look at the state.
- Interposing
vforkno longer corrupts the target (a recorded boundary, then a guaranteed tail call). - Boundary tolerance (trace contract v3, ADR 0002): a fork/spawn boundary is explorable when an oracle is present and no process other than the subject touched the state directory. Everything else refuses with a named detector, and any boundary without an oracle is UNKNOWN.
Acceptance, measured: six acceptance cases over one binary with one environment variable of difference decide tolerance by the child's behaviour alone, and the dogfood target now travels past the boundary gate to fail for its true reason (a rustix-issued raw syscall — #19) instead of the categorical one.
Goal: the three-commands-and-a-directory contract of DESIGN §12, complete.
Shipped, one PR per piece (#43, #44/ADR 0007, #47/ADR 0008, #49/ADR 0009, plus the acceptance):
- Refusals name the operation they refused on (#41): the divergence index, the raw strace line, the shim's account — in text and JSON alike.
sideeye.toml(ADR 0007): a hand-parsed strict subset owning the define surface only; unknown keys refuse with their line, and a key joins the schema only in the change that enforces it.- L1 success markers and post-success invariants (ADR 0008): judged against the whole post snapshot in worlds where the marker reached stdout before the kill; a marker the clean run cannot produce refuses (
marker_never_observed) instead of going silently vacuous. - L2 checker scripts and checker falsification before every run — both shipped early, in v0.1, and exercised by every dogfood since.
- Case storage with landing context and
sideeye replay(ADR 0009): replay is the explore pipeline restricted to the case's crash point plus the baseline, every trust gate intact; a changed recording answerscase_no_longer_applies, never a verdict about a shifted address. - Scope narrowed, deliberately: shrinking in v0.3 means the earliest failing crash point, reported with its logical address. "Simplest" and measured reproducibility counts remain future work, and the report claims neither.
Acceptance (measured, in spike/acceptance.sh):
- The doctor-cross-examination scenario runs end-to-end against a toy target, driven by the toml alone (check 2aa).
- Conditional-invariant vacuity both ways (check 2y): the marker-observed worlds are a strict subset of the crash worlds (
0 < 4 < 8measured on the marker toy), an unflushed marker yields an honestly vacuous zero, and a marker the clean run cannot produce is UNKNOWN. - The define budget, measured on a fresh target (
spike/dogfood-watson/): watson is driven to a correct, named refusal —baseline_violates_invariant, its frames carry a fresh uuid per run — by asideeye.toml, one checker script, and one environment variable. The budget held; the refusal did its job.
Goal: the second platform DESIGN §9 committed to, with honest edges.
Delivered in v0.1, ahead of its slot: macOS interposition with an identical CLI and contract, and a parity assertion in CI (identical scenario, identical verdict and crash point on both OSes). The original scope's last honesty gap closed 2026-08-18 (#10): docs/target-classes.md states plainly that a macOS target must be self-built or self-installed, never an Apple-shipped binary, and the report's macOS build names an Apple-shipped platform binary as one possible cause on the no_shim_marker line — with the refusal shape pinned in the macOS CI job. No longer a numbered milestone.
The original scope, for the record:
- macOS interposition; identical CLI and contract.
- Hardened-runtime / library-validation targets detected and reported as unsupported (exit 2) — never silently mis-tested. Same for statically linked Linux binaries.
- A parity suite: identical scenarios must produce identical verdicts and equivalent crash points on both OSes.
Acceptance: parity suite green on macOS and Linux; unsupported-target detection falsified once per OS.
Goal: evaluate the primary success criterion (DESIGN §17) for real.
Scope:
- Run Sideeye against omamori's stateful operations (its state is file-based; it qualifies under DESIGN §9 constraints).
- Regression-case stability in practice: saved cases replayed across real code changes; context mismatch must produce "case no longer applies", not a silent pass.
Acceptance: DESIGN §17's primary criterion evaluated honestly — either a novel, author-confirmed crash-consistency bug, or a written analysis of why none was found. The analysis feeds the kill criteria (DESIGN §18); this step cannot be skipped or softened.
Status (2026-08-12, updated same day): calibration kill condition cleared; regression-case stability measured in practice; the full §17 primary criterion is not yet met. The second scope item is now measured rather than argued (spike/dogfood-timew-replay.sh, BUILDLOG same date): the saved timewarrior case replays FAIL on the unpatched pinned build, replays PASS across the applied fix rebuilt under the same command name (explored 2, landing context intact), and answers case_no_longer_applies — a refusal, not a verdict — against a build whose recording differs (distro 1.4.3, 19 ops vs 24). The state-mutating omamori subcommand this evaluation drove — exec — was reconnoitred (BUILDLOG, same date): the write pattern confirms the high-water mark after the body, so verify stays conservative in every crash window and no §17-class bug exists on that path — a clean instance of "too hardened" (§18), the survivable side. The guarded self-modification commands (config-modify, init --force, audit key rotate) refuse for a human at a terminal exactly as for an agent (#12); measuring one as a Sideeye operation would require break-glass, which removes the defence under test, so it is out of scope on discipline (the full surface was enumerated the same day — BUILDLOG: every subcommand including nested arms, the argv0 shim mode and the hook entrypoints, cross-checked from the write-primitive side; of the four unguarded writers beyond exec, install/setup/init refused at named trace-contract walls under v8 — symlinkat, fchmodat — until #141 re-measured them under v10 (2026-08-16): all four explore fully and PASS, audit verify's high-water-mark bootstrap write included (4 crash points in that run; counts move with image and contract); the one non-atomic write found, retention's in-place prune rewrite on the append path, is recorded there as an open finding that needs clock control to drive). The find is on the calibration target §18 required: timewarrior, no hand-written adversarial tests, whose crash-window bug (timew undo destroying committed data, timewarrior#778) is the mechanical inverse of omamori's safe ordering. That clears §18's calibration kill condition. It does not yet satisfy v1.0's entry criterion 1: of §17's six conditions, four are clean and two have gaps — "discovered automatically" holds only for the crash-world search (manual trace triage seeded the target and window; DESIGN §17 status note), and the finding is kept as a reproducible recipe, not a sideeye replay regression case (it needs a built timewarrior). (Updated 2026-08-16: the regression gap is closed as hygiene — the timew-regression CI job runs the recipe's record/FAIL/PASS legs on every push to main and every pull request (#82) — and the discovery gap is superseded rather than closed: ADR 0017 redesigned criterion 1 around provenance, and the timewarrior finding stays outside it by the ordering requirement.)
Goal: prove the counterexample is agent food, and calibrate detection power.
Scope:
- The report JSON documented as a schema; a quickstart for CI (GitHub Actions example).
- Loop-closure test (DESIGN §17, second criterion): a coding agent receives only the report and the repository, and must produce a fix that makes the replay pass — no human translation.
- Calibration target (DESIGN §18): at least one deliberately average stateful CLI with no hand-written adversarial tests, explored with honest results.
Acceptance: loop closure demonstrated end-to-end at least once; calibration results published in the buildlog, whatever they are.
Status (2026-08-12, superseded the next day — see below): the agent-facing surface is built — and shipped early, in v0.4.0 (the adapter landed on main before v0.4 was cut, so the 0.4.0 tag carries it; this milestone stays open and cuts 0.5.0 when its own acceptance is met). sideeye mcp (ADR 0010) gives an agent the standard MCP tool surface — explore a sideeye.toml and replay a saved case, both measured end-to-end over the wire (explore → case → replay reproduces). The calibration target is already met (timewarrior, §18/§17 status). What remains for the milestone is the loop-closure test itself: hand an agent only the report and the repository through this surface and see whether it fixes the finding without human translation — the surface is ready to run it.
Status (2026-08-13): loop closure demonstrated end-to-end — the milestone's
acceptance sentence is met. A context-free coding agent, handed only the
counterexample (the report JSON, the case it names, the declared invariant it
points at, bug-blind replay plumbing) and the pinned timewarrior checkout,
re-derived the fix and the judge's own replay passed, with the feature intact
and the audit clean (spike/loop-closure-timew/; BUILDLOG 2026-08-13, "Loop closure: the
protocol is fixed before the run", which quotes the run's manifest — written under
spike/runs/, which is not committed (#825)). v1.0 entry criterion 2 is met by
this measurement. Two honesty notes: the first run drove the CLI replay
surface through the plumbing script, not the MCP adapter — resolved the same
day: attempting the MCP-mediated variant first surfaced and fixed two real
surface gaps (ADR 0011 — the child env dropped TIMEWARRIORDB-class
variables, and a persistent server had no per-call state freshness), and the
confirmation run then closed the loop through this surface with a second
model (claude-fable-5, BUILDLOG 2026-08-13, "The MCP-mediated run", quoting its uncommitted manifest —
loop_closed true, all three controls held, audit clean); and the input set is
the report and what it transitively names, not "the report alone" (the
exact set is declared in the BUILDLOG protocol). The milestone's remaining
scope closed the same day: the report is documented as a schema
(docs/report-schema.md, held to the generated reports by acceptance
check 4), and the CI quickstart is a real workflow this repo runs
(docs/ci-quickstart.md). Nothing of v0.5's scope remains; 0.5.0 released
2026-08-13.
Entry criteria — all must hold, none may be argued around:
- Primary success criterion met: a real, novel, author-confirmed crash-consistency bug discovered by Sideeye's deterministic judge from an invariant declared and committed before this project observed any failure of the target in execution (reading a report of a failure while scouting is not observing one) — with the question's provenance recorded and labeled: blind (posed under ADR 0012's two-seal protocol) or assisted (a scout read the target; the scout and its sources named) — fixed, and kept as a replayed regression case. "Novel" requires a recorded tracker search with a positive control, as campaign 1 scored it; "author-confirmed" reads as §17 scored it for timewarrior: this project's author judges the bug real, upstream confirmation is sought, not required. An assisted finding is never presented as blind; blind remains the stronger provenance and nothing assisted may borrow it, and the ordering requirement holds timewarrior's "discovered automatically — partial" scoring exactly where §17 put it (its checker was written after the failure was observed). Redesigned 2026-08-15 (ADR 0017, which carries the full argument, the re-scoring of past findings, and the costs): the redesign path was pre-committed in #118 after the campaign nulls were known and before any assisted evidence existed, and the #118 experiment then measured that the binding constraint on reaching a verdict was the judge's reach, not the question's provenance (
spike/assisted/REMEASURE.md). - Loop-closure criterion met: an agent fixed a finding from the report alone.
- Kill criteria (DESIGN §18) reviewed against collected data; none triggered.
- UNKNOWN rate on supported targets measured and published; a target threshold is set from that data (not invented in advance) and met.
- Contract frozen: config format, report schema, exit codes, replay compatibility, the MCP surface (added by decision, 2026-08-13; the normative declaration is
docs/contract-freeze.md). - Docs: a fresh machine reaches its first exploration in under ten minutes from the README.
If criterion 1 cannot be met, v1.0 does not ship — the kill analysis ships instead.
Criterion 1 status (2026-08-25): met — the last open condition adjudicated by the owner (#305). The himalaya finding (cohort 4) already held the rest on the record the criterion-1 trail below carries: novel on the recorded tracker scan, discovered by the deterministic judge from a define whose precedence over any exploration artifact the mini-seal checks mechanically, provenance recorded and labeled, author-confirmed in the criterion's own wording (confirmation sought and received — upstream fixed the bug the day it was reported and closed the report), and fixed, in io-maildir 0.3.1. What remained was "kept as a replayed regression case": the fix adds a staging rename, a case pins a crash point inside a recorded operation sequence, so the exhibit refuses against the fixed build (case_no_longer_applies, the freeze's promised behaviour) and the timewarrior-shaped "PASS patched" leg is unreachable by construction. The ruling accepts a second shape in exactly that circumstance: the committed exhibit replays against the pinned build that carries the bug (positive control reproducing the committed transcript — empty diff after normalising the minted filename, the per-run work path and the driver's own header lines), and the defect's absence on the fixed build is measured rather than inferred — a fresh explore passes 4/4 through an instrument that relaxes exactly one assertion of the frozen checker (the guard whose stated premise the fix falsified) and otherwise only records evidence without judging it, with the checker still falsified before the run, a negative control showing the relaxation leaves the original defect caught on the stock build, and a functional control pinning that a complete copy actually occurs (spike/cohort4/himalaya-r2/upstream-fix/). The full ruling, and what it deliberately gives up, is recorded with the criterion-1 trail below. Instrument note (2026-08-26, trace contract v11): every measurement in this paragraph was taken under contract v10, and the replay leg cannot be re-run as written from v11 on — the committed exhibit is a v10 case, so a current engine answers case_no_longer_applies, which is the freeze's promised behaviour rather than a regression (docs/contract-freeze.md surface 4). Re-running it needs the engine of that day. The ruling stands on measurements already taken; what changed is that they are now dated rather than reproducible in place. The apparatus that define carries (no-accel-copy.so, which answered the kernel copy primitives in userspace) is also superseded: v11 interposes copy_file_range and sendfile directly (#244), so the wall it was built to route around no longer stands.
Criterion 4 status (2026-08-16): met. The measurement is docs/unknown-rate.md — a two-group corpus frozen and merged before the sweep ran (apparatus PR #142, then the results PR; the first-parent order is the audit trail). The A-group is published as the engine's development-input set, deliberately not the threshold basis, and it has now been measured twice: generation g1 (2026-08-16) at 1/28 UNKNOWN (3.6%, watson's known nondeterministic-writer refusal) and generation g2 (2026-08-26) at 2/36 (5.6%), after #239 sorted the eighteen cohort defines committed since g1 across three ledgers — eight into the corpus, six superseded by later revisions of the same targets, four outside the supported classes. Both figures stay published with their dates. Where the movement sits is measured; what caused it is only partly separable. g2 re-ran the twenty-eight trials g1 covered and every one produced an identical verdict and reason on a newer engine against rebuilt images, so the shared subset reads 1/28 in both generations and the difference is located entirely in the eight trials that entered at g2. That is not the same as attributing it to their being new: no run exists in which those eight faced g1's engine, so their novelty and the engine change cannot be separated from each other. The eight moved the rate in both directions: one added to the numerator and seven added only to the denominator, so the shared twenty-eight plus himalaya alone would read 2/29 (6.9%) and the other seven bring it to 2/36 (5.6%). The added UNKNOWN is himalaya, whose define carries apparatus_superseded — its no-accel-copy.so answers the kernel copy primitives the shim interposes as of #244, and the collision reports oracle_saw_phantom. That one refused for an engine reason, on a define that was new to the corpus. The B-group — 20 never-run targets machine-selected from Debian's own package metadata, 7 reaching the uniform define stage — measured 3/7 UNKNOWN (42.9%), every one a define-budget refusal with a named reason: four of the five targets whose invocations could be spelled as operation strings reached verdicts (including a fresh FAIL on bogofilter-sqlite, disposition pending triage), and the fifth (cookietool) was refused on the exit convention the uniform protocol declared, not on spelling. The threshold, set from that data by the owner and recorded with it — with the page disclosing that the origin classification postdates the sweep and that cookietool is its arguable case: target-origin UNKNOWNs ≤ 1/7 (measured 0; 1/7 even if cookietool is re-filed) and overall ≤ 50% (measured 42.9%) — both hold. The generated tables are held to the committed reports by acceptance check 12 (the surrounding prose is held by review, not by the gate — with one exception since #342: the ledger counts in the sentence above, and the seven figures in the three sentences on that page which sort the same defines, are recomputed from the ledgers and the disk by spike/check-ledger-prose.sh — eleven in all, read from both pages); the funnel (20 selected → 13 documented walls → 7 explored) and the per-axis slices are on the page.
Criterion 4 status (2026-09-18): the B-group was re-measured on the released v1.5.0 (generation g3, #619) and held to the frozen threshold; whether the status moves is the owner's call, and is not scored here. g3 reads 2/7 UNKNOWN on the B-group (g1: 3/7): five trials identical, hnb moved from a refusal to FAIL and lbdb from one refusal to another, neither by a define change (the engine and the image both changed). Part 2 holds at 28.6%. Part 1 depends on how lbdb's child_touched_state_dir is filed, and the page records both readings without choosing: as target-origin (the page's own September wording) it is 1/7 and holds at the edge, with g1's robustness note gone — cookietool re-filed as target-origin would make it 2/7; as a spelling-and-mode gap (the same operation reaches PASS under --observe syscalls, docs/target-classes.md) it is 0/7, as g1 filed it. The same sweep measured B2 — thirty never-run targets machine-selected from Debian 13 after v1.5 shipped, ADR 0073 — at 5/19 UNKNOWN (26.3%) past eleven walls, published beside B and never pooled with it; no threshold is set from B2. The record is docs/unknown-rate.md, generation g3.
Criterion 5 status (2026-08-18): met. The pre-tag re-sweep ran: snapshot replaced (13 open issues, all classified), the four issues filed and resolved inside the inter-sweep window accounted (#169's world-only boundary refusal — class A, fixed; #167's UTF-8 defang, fixed; #164, fixed with #27's measurement; #165, its accidental duplicate) with #159's held call resolved alongside, and the freeze declaration moved to its permanent home, docs/contract-freeze.md, which carries all five surfaces normatively (the four this criterion names plus the MCP surface decided 2026-08-13). #86 closes with the re-sweep's merge. The audit is docs/freeze-audit.md — every open issue classified against the five frozen surfaces across two sweeps (twenty-six at the original, 2026-08-17; thirteen at the re-sweep), the classification held to a committed snapshot by a self-falsifying gate, and every class-A adjudication executed (fix: #46, #164, #169; demote: #5; measured already-fixed: #27; narrow: #39 — its issue stays open as the mkstemp family's lookout post; the A-adjacent #10 and the class-B #150 landed with their records — none left as a documented hole under an intact PASS claim). The exit-code split from #94 is rejected permanently, with reasons on the declaration. (Superseded 2026-08-27 — the third sweep's line below is the current state; this line is kept as the record of what was measured and ruled on 2026-08-18.) (The 2026-08-17 fix list corrected and then emptied the same day: #27 measured already-fixed by #122's pair rule with the adjacent #164 surfaced and closed by that measurement; #46 landed with per-world arming; #5 landed as the adjudicated demotion, whose review first forced the DT_UNKNOWN classification repair; #58 fixed ahead of its deferral by owner decision. #10's narrowing and #150's relabel landed 2026-08-18.)
Criterion 5 status (2026-08-27): the third sweep ran; whether the criterion is still met is the owner's call, and is not scored here (#281, #353). (Superseded 2026-08-29 — the ruling is recorded in the line below; this line is the state as it stood on 2026-08-27.) The sweep replaced the snapshot (55 open issues, up from 13), classified 113 rows across both tables, accounted the 38 issues opened and closed inside the window by a closed-issue query, and migrated the 17 historical resolved rows into the manifest so the whole page is generated from one trust root. Its accounting is recomputable from a committed acquisition (spike/freeze-audit/capture-2026-08-27-raw.json) rather than quoted from a transcript.
Two findings bear on the criterion and neither was visible to the previous sweeps. First, the five surfaces were measured rather than read for the first time (ADR 0028): the unknown_reason closed set gained five members inside the window, the report schema gained one field, and contract_version went 10 → 12. All are legal — the closed-set additions because they landed before the tag, the field under surface 2's additive allowance, the bumps because surface 4 says a bump is not a broken promise — but the audit had recorded none of them, and one of them (trace_too_large, from #324) is attributable only to a resolution, since the issue's own text names neither the field nor the set. Second, the declaration moved three times inside the window (0e035eb, 9f04932, 975e2fd), including the additive allowance most of this sweep's readings rest on, which was written eight days after the snapshot being replaced was taken, and a correction to surface 4 whose previous text was measured against the code and found wrong. The sweep now pins the declaration's revision, and the gate reports it as staleness (exit 3, alongside surface drift) if it moves — it failed with exit 1 until 2026-08-30, which stopped every leg behind it (#371).
What the criterion's status turns on is a three-item list this sweep put back on the page, where the previous two sweeps left it empty: #263 (a per-world timeout), #323 (a tree-total cap) and #363 (eight refusals that exit 3 after the define has run). Each would need a new unknown_reason member, which the tag forecloses because the closed set is excluded from the additive allowance; two of the three are conditional by their own wording, since reusing an existing member carries no deadline. Class A is unchanged and clean — eleven rows, every one of them adjudicated and its ruling executed (ten closed at this sweep; #39 stayed open as the mkstemp family's lookout post with its narrowing executed on 2026-08-17, which is what the invariant's first clause is for, and closed on 2026-08-31 when the class stopped being a wall — ADR 0036), and not one left as a documented hole under an intact PASS claim — and no clause of the recorded invariant is violated by an undecided issue. Whether the freeze should proceed with three foreclosed decisions outstanding is a judgement about the release, not about the audit, and this line records the material rather than the verdict, the same way criterion 3's reopened line did before its adjudication.
Criterion 5 status (2026-08-29): met — ruled by the owner on the fourth sweep's record. The three-item list the 2026-08-27 line left open has emptied the only way the closed set's exemption permits: every foreclosed decision was taken and its member landed before the tag — state_tree_too_large (#323, 2026-08-27), state_rewrite_failed plus three exit paths moved from 3 to 2 under the declaration's existing clause (#363, 2026-08-28), and child_timed_out (#263's first half, 2026-08-28; the issue stays open on its stdin question, which touches nothing frozen). Each landing was measured into surface-changes.tsv through the gate's own drift path when it happened; the fourth sweep (snapshot of 56, window 2026-08-27..29) found zero issues opened-and-closed inside the window, classified the nine filed since — every forecast none — and closed with the offline gate green, --live clean, and docs/contract-freeze.md byte-identical across the window. Class A holds eleven rows, all executed, none resolved by prose over an intact PASS claim. The standing obligation is unchanged: whatever is filed between now and the tag gets swept before it, and check-freeze-audit.sh --live runs immediately before the tag and must be green.
Criterion 6 status (2026-08-17): met, first measurement. The protocol was committed before the clock ran (spike/onboarding-clock/PROTOCOL.md: a network-off fresh container, the README as the only sideeye documentation, a context-free headless driver not told it was timed, the external target jrnl chosen by the owner under the standing selection bar). Wall-clock: 4 minutes 22 seconds from the first timestamped event to explore returning a real PASS on jrnl (the init event carried none, so this is the first assistant turn — disclosed in RESULTS.md from the start, and stated precisely here since 2026-08-28) — 4 of 4 explored worlds (3 crash points + the baseline), oracle agreeing, checker falsified first — derived from the committed event timeline, not hand-written (spike/onboarding-clock/RESULTS.md, which also records the audit's three flagged transfer commands and their adjudication). The rehearsal that preceded the clock found the aarch64-linux release artifacts of v0.9.0 and v0.10.0 broken on lesser CPUs than their builders' — measured on Apple-Silicon Docker; the x86_64 artifact shares the construction, presumed affected and unverified (fixed and repaired the same day — the release matrix now pins -Dtarget); the run measured the repaired artifact. One run is one measurement: re-runs after doc changes append under the same protocol, and the last run before the freeze is the criterion's evidence.
Standing note (2026-08-23): run 1 is still the only run, so by the rule above it is still the evidence — but the document it measured has moved underneath it. Seven commits have touched README.md since that day (git log --since=2026-08-17 -- README.md, counted 2026-08-23) and this documentation change is the eighth: the install section now leads with the Homebrew formula (#180, #282), the version strings advanced twice, the refusal-flag wording was corrected (#159, #167), and the documentation table gained the evaluation pages. All of that sits inside what the protocol hands the driver as its only sideeye documentation, so what run 1 timed is not what a fresh reader opens today. The criterion is not re-scored here — re-scoring a met criterion inside a documentation change is the move this repository refuses — but a re-run is owed before the freeze, and the instrument hardening it needs first is #160.
Standing note (2026-08-28): still one run, and the count of measurements standing under the current protocol is now zero — the protocol has been amended twice since run 1, and its own rule restarts the count on each. Thirteen commits have touched README.md since run 1 (git log --since=2026-08-17 -- README.md, the same command as the note above, counted 2026-08-28); against the blob the driver actually read, the file differs by +31/−14 lines. #160 is closed, but the hardening it delivered did not reach the audit's other two predicates, and the one precondition it named for run 2 — probing the declared permission scope from a plain terminal — was run on 2026-08-28 and came back negative: --allowedTools grants rather than confines, and denying one execution tool relocates execution instead of ending it. The audit was enumerating what to reject, so a tool on neither list was invisible to it; that instrument is rebuilt in the same-dated protocol amendment, which is what run 2 now waits on rather than #160. Two consequences for anything previously said about run 1: its allow-set was the bare Bash, so it is not evidence about the scoped pattern either way; and its committed timeline records one row per event, so a tool census taken from that file counts the last content block of each event and not the calls made — whether run 1's driver stayed inside its allow-set is unmeasured, and the raw transcript is not committed, so it is no longer measurable. Its wall-clock arithmetic is unaffected. The criterion is not re-scored here — the same rule as the note above, and none of this is a measurement of the documentation.
Standing note (2026-08-28, later the same day — supersedes the count in the note above): run 2 was taken, and it is met at 2 minutes 55.707 seconds. By this page's own rule — re-runs append and the last run before the freeze is the criterion's evidence — run 2 is now that evidence, and run 1's 4:22 is the record of its own date. The verdict is unchanged, so this is a measurement record rather than a re-score; what changes is which run the criterion points at. Derived from the committed event timeline (spike/onboarding-clock/RESULTS.md, which carries the full adjudication), against the README at this commit, with the checker falsified in the same output that returned PASS over 4 of 4 explored worlds. Two things a reader should carry from it. The container was not newly created — docker run hit a name conflict and the run proceeded against one started two hours and fifty-one minutes earlier; measured rather than assumed, zero files inside it changed in that window against a control of 47 during the run, and its README, tarball, target and target configuration are byte-identical to a fresh container's, so the run stands with the deviation on the page. And the run's own evidence now carries what the same-dated protocol amendment was written about: the CLI granted 18 tools where the launcher declared 6, the twelve extra including the one an earlier probe used to run a host command, while every one of the 11 denied names was absent. The driver used only Bash, 23 times. Two numbers under two protocols, two artifacts and two prompts are two measurements, not a series.
Standing note (2026-08-30): the README moved again — the MCP section gained an environment table, a schema date, and two runnable blocks (#389). Run 2 remains the evidence and its time is unchanged; this is the same disclosure the 08-23 and 08-28 notes make, that the document the driver read is not byte-identical to the one on this page. Two things narrow it here. The driver's task is the CLI path — tarball to a verdict on an external tool — and the section that moved is the one a driver reaches only if it chooses the MCP route, which neither run did. And the change is additive within one section: the paragraph the earlier runs read is unchanged, sitting after the new subsection. What is new, and is the reason this note is shorter than its predecessors, is that part of the README is now executed on every pull request and every push to main (spike/check-readme-mcp-call.sh) — for that part, drift between the page and the product is a red check rather than a note.
Standing note (2026-08-31): the apparatus changed, and the run count deliberately did not restart — run 2 remains the evidence at 2 minutes 55.707 seconds. run-clock.sh now builds the image and creates the container itself, and refuses when one of that name already exists in any state (#383); meta.json gains a box block naming the container, its image, its creation time and the sha256 of the Dockerfile it came from. This is named here because a reader who knows this protocol's rule — a change to it restarts the count — would otherwise assume it did. The owner ruled on 2026-08-31 that it does not, and the distinction is written into the rule rather than applied from outside: the count restarts when a change moves what a run's number is attributable to (the measured document, the stimulus, the artifact under test), and a change to what the operator types moves none of them. The driver reads the same README, gets the same prompt and tools, and works in a box with the same contents built from the same file. Two supports this ruling explicitly does not use: that each earlier restart was triggered by a stimulus change (they were bundles of five and ten items, mostly instrument-only, and which item forced the restart is not recoverable), and novelty (the second 2026-08-25 amendment already declined a restart in as many words). Run 2's stale-box deviation is unaffected and stays on its own page — had this check existed it would have refused that launch, which does not retroactively place run 2 under a different protocol, because the sentence it departed from is the same sentence in force now. What changed is that the next departure is a refusal instead of a disclosure.
Standing note (2026-09-12): run 2 is still the evidence at 2 minutes 55.707 seconds, and the document it measured has moved much further than any earlier note records. Thirty commits have touched README.md since run 2 (git log --since=2026-08-28 --oneline -- README.md, counted 2026-09-12) and this documentation change is the thirty-first: the page goes from 238 lines and 33,714 bytes to 111 and 8,365, with the flag list and the worked example moving to docs/cli.md, the MCP surface to docs/mcp.md, and four constraints' detail into DESIGN's constraint list. That is the largest single movement in what the protocol hands the driver as its only sideeye documentation, and the box carries no docs/, so everything that moved is unreachable there — the protocol's own reading of that is recorded in RESULTS.md: dead relative links in the box are an artifact of the box, not a gap in the page, because a real reader has them. The criterion is not re-scored here — the same rule as the notes above. What this change adds instead is a mechanical floor under it: spike/check-readme-shape.sh fails the build if the page loses any of the seven things run 1's driver named as load-bearing, or grows past the sum of what it has to carry. A check that the page still says those things is not a measurement that a fresh machine reaches a verdict from it, so the re-run remains owed, and it is owed against a page that is 58% of the size run 2 read (8,365 bytes against the 14,424 its clock_start commit carried; the quarter-size comparison is against HEAD, which neither run read).
Criterion 3 status (2026-08-16): met. The review is docs/kill-criteria-review.md — every §18 condition quoted, scored against named committed evidence, with the counter-evidence and margins on the page. Six rows are scored on measurements (the assisted cohort, the #84 sweep, the loop-closure test, the #144 follow-up; the calibration paragraph stands on #141); row 2 is scored on the recorded absence of the very comparison it asserts — no measurement supports the condition, and the page names what would; row 7 — the UX-difference condition, the one row whose wording makes absence of data a trigger rather than a neutral — is recorded as an owner adjudication (2026-08-16: not triggered), with the missing head-to-head disclosed on the page rather than argued around. The page also discloses that most rows draw on a single measurement family, so the review is one instrument read eight ways, not eight independent confirmations. None triggered. Acceptance check 11 holds the page's repository paths against rot; the numbers are quotations whose recomputation lives at their source pages. Per the page's closing rule, any future measurement landing on a row's trigger side reopens the review — the rows do not close permanently.
Criterion 3 current status (2026-08-26): reopened — re-score pending owner adjudication (#240). The dated line above records what was measured and ruled on 2026-08-16 and is kept as that record; this line is the current state of the gate. The three selection cohorts' twelve targets were entered against all eight rows in a dated re-review section, and two of them — row 4 (environment artifacts) and row 6 (setup weight) — carry evidence on their trigger side, so the rule above has fired. The status on this line is left as it was measured and dated: no verdict was moved and this criterion was not re-scored. What the re-review supplies is the material for a re-score; the adjudication, and its consequence under the page's rule that a reopened row means the analysis ships instead of the release, is the owner's and is not recorded yet. (Superseded 2026-08-27 — the adjudication is recorded in the line below; this line is the state as it stood on 2026-08-26.)
Criterion 3 status (2026-08-27): met — the two reopened rows adjudicated by the owner (#240). Neither row 4 (false positives / environment artifacts) nor row 6 (setup weight) is triggered, so the reopen closes without the release gate moving and no §18 analysis ships in place of v1.0. Row 4's case is recorded as a second example of the existing #35 class ("tools with non-durable scratch files") rather than a new class, with the new property that a target-created scratch path can be relocated into the judged root by the measurement setup; row 6's ruling is that declared apparatus is instrument cost and not setup, bounded so that provisioning what the target depends on stays setup, and that define-revision cost is excluded from row 6 and left unallocated rather than reassigned. The binding definition lives in DESIGN.md §18 — criterion 3's conditions are defined there and this line points at them rather than restating them — and docs/kill-criteria-review.md's adjudication section carries the rejected readings and what each ruling gives up. The two dated lines above are kept as the record of what was ruled on 2026-08-16 and what was reopened on 2026-08-26; the rows still do not close permanently.
The status line above, with the marker beside it, is the one current status of criterion 3; the dated state markers it cites are on docs/kill-criteria-review.md, the definition in DESIGN.md §18, and spike/check-criterion3-status.sh holds the three to that split (#356). v1.0.0 shipped on 2026-08-29; what this guards is the review page's closing rule — no row closes permanently — for whatever release comes next.
Criterion 1 status (2026-08-14, after the blind campaign on topydo 0.14). Three of its legs are now closed and three remain, and the split is deliberate rather than rhetorical:
- found by Sideeye — closed, blind. Twelve of thirteen declared operation forms produced a counterexample from an invariant committed before any crash world of that target existed.
verify-sealsreturns ALL SEAL CHECKS PASSED (R1 audited) against Seal B5a034aff, so the ordering is machine-checked, not asserted (DESIGN §17, and the three honesty bounds it carries). - kept as a replayed regression case — closed. Each saved case replays inside the pinned image with no target build (
exit 1,the case reproduced). This is the shape#82asks for; the timewarrior recipe could not provide it. - novel — measured 2026-08-14 (tracker search recorded in the campaign ledger), and the answer is split. The blind-found crash-window destruction is not novel as a phenomenon:
topydo/topydo#318reports the same failure surface (disk-full write destroying the list) — the campaign adds mechanism and a replayable counterexample, not the discovery. The recovery misfire (post-crashrevertundoing an older command; contradictory documentation on the matching rule) was not found in the tracker and is novel as far as that search sees — but it came from post-seal analysis, not the blind search. Consequence: no single finding currently satisfies "found by Sideeye" and "novel" at once, and criterion 1 stays open on that conjunction. Ruled 2026-08-14 (author): the misfire does not count. The scale that scored the timewarrior find partial — a human formed the specific hypothesis — reads the same here with the arrows reversed, and the sealed declaration itself pre-committed to calling recovery measurements analysis, not findings. Criterion 1 therefore waits for another blind target (superseded 2026-08-15 by ADR 0017 — the criterion no longer gates on blind provenance; the rulings in this paragraph stand): the designated path was a second campaign under fresh seals whose declaration includes recovery-path invariants ("after a crash plus the documented recovery command, intact data survives" — declarable from documentation alone), which is the lesson this campaign paid for. Timing is a resourcing decision, deliberately not fixed here. The misfire keeps the value it already delivered upstream (topydo/topydo#341); it simply does not wear the "discovered automatically" badge. - real / author-confirmed — open. Not asserted by the run. The sharpest observed behavior lives in the documented recovery path and is post-seal analysis, not automated discovery;
spike/blind-hunt/analysis/findings.mdkeeps the halves apart. - fixed — open. Not attempted.
The second campaign ran (2026-08-14, campaign 2 under ADR 0015): null result on abook 0.6.1, and criterion 1 stays open. khard — first in the inherited order — was burned before Seal B: its red suite let the checker's leading query run over deliberately mis-shaped stores, committing the target's failure response pre-seal (the burn, the machine reselection, and the full disclosure are in the campaign ledger and PR #110). The sealed predicate then selected abook. Its declaration — three --convert forms including the observed refusal as an expected_status = "1" operation, the recovery-path rule discharged vacuously because the documentation names no recovery command at all — was machine-verified to precede exploration: ALL SEAL CHECKS PASSED (R1 audited) (the committed transcript spike/blind-hunt2/analysis/verify-seals.txt), exploration head == Seal B in a clean tree, engine and shim byte-equal to the sweep's (R3). Every crash world satisfied the declared invariants (import 2+1 worlds, export 2+1, refused 1+1; violations 0), with the engine's falsification gate proving the checker red on corrupted state after the seal (spike/blind-hunt2/analysis/findings.md). A null campaign is the protocol working, not failing — the same honesty that scored timewarrior partial and the misfire not counted also records "nothing found" without inflating it. Remaining unconsumed candidates in the sealed order: khal; hledger refused at sweep (reason sealed, unread). Whether to run a third campaign is a resourcing decision, deliberately not fixed here.
The third campaign ran (2026-08-14, campaign 3 under ADR 0016): null result on khal 0.14.0, and criterion 1 stays open. The inherited order (khard burned, abook consumed) selected khal by the sealed predicate over a fresh sweep (khal 0 / hledger 2, reason still sealed unread). Its declaration — import of a fixed-UID .ics as the live search, import-update and new with pre-registered refusal expectations, the recovery-path rule vacuous over the full help set plus the version-pinned usage page — was machine-verified to precede exploration: ALL SEAL CHECKS PASSED (R1 audited) (spike/blind-hunt3/analysis/verify-seals.txt). Every crash world satisfied the declared invariants — import 10+1 worlds, update 21+1, new 10+1, violations 0 — and both pre-registered refusals did not fire: the recordings were accepted and explored in full, which is wider coverage than the declaration promised itself. The vdir/iCalendar storage-class disclosure (todoman, explored by this project) is discharged in the declaration and the findings. Remaining unconsumed candidate: hledger, unselectable while its sweep refusal stands; understanding that refusal means unsealing it — deliberately not done. Two designated-path campaigns have now returned null. Whether criterion 1's remaining path is more campaigns, a different target class, or §18's kill analysis is a resourcing and judgement decision, deliberately not made in a status paragraph.
Criterion 1 redesigned (2026-08-15, ADR 0017): provenance is labeled, not gated. The sealed candidate pool was run to exhaustion first — one non-novel find (topydo), two nulls (abook, khal), the last sealed candidate unselectable behind an unread refusal; a fourth campaign under fresh seals remains possible per ADR 0012 and declining it is a resourcing judgement. The assisted experiment (#118, its scoring rules pre-committed in the issue after the nulls were known and before any assisted run) recorded an inversion: question quality 5/5, the binding constraint the judge's reach. The three measured syscall gaps closed the next day (#121, #122; the fourth gap, exec, is #123 and deliberately deferred), and the same committed defines reached verified, replay-confirmed counterexamples on stow, devtodo and buku — calcurse was verified before the gaps closed and is the control (spike/assisted/REMEASURE.md). The criterion now requires the judge's discovery from an invariant committed before any observed failure, with provenance labeled; novel/author-confirmed/fixed/replayed are unchanged and all still open — novelty is deliberately unchecked for all four assisted findings, and the tracker searches are the designated next step (they ran the same day; next paragraph).
The assisted findings' next legs ran (2026-08-15): novelty four-for-four, two upstream reports standing, buku withdrawn. The recorded tracker searches (spike/assisted/NOVELTY.md, positive controls throughout) found none of the four previously reported. The report step then applied two harder gates in sequence: reproducibility with tooling a maintainer already has (strace fault injection), and fairness at target selection (now a PROTOCOL.md selection-time rule). Outcome: calcurse and stow reported and standing (lfos/calcurse#529, aspiers/stow#139); devtodo's report filed and withdrawn on the fairness rule, its finding staying in-repo; and buku's finding withdrawn entirely — its one remaining leg was a misread of the falsification gate's output, and buku's own recovery-open succeeds in every crash world (spike/assisted/buku/RUNLOG.md, Correction section). The count of assisted findings carrying a live contract-level claim is therefore three (stow, devtodo, calcurse); buku remains a verified engine-level L0 observation sitting inside sqlite's documented recovery contract. Author-confirmed / fixed / replayed remain open on all three.
The second selection cohort ran (2026-08-21, #183): five frozen targets, six recorded outcomes, and criterion 1 stays open. Its question was what happens when Sideeye meets mature software carrying its own transaction machinery. Two probe walls (Borg on determinism, later lifted by declared apparatus under #200; KeePassXC on determinism and on seven mutating calls that could not be attributed to a path), two engine walls (Jujutsu — no_shim_marker, a statically linked release binary; Bun — multiple_threads_detected), and two full verdicts that are the interesting half: Mercurial FAIL 73/107 and Borg FAIL 3/119, every violation in both L0-only, with each tool's documented contract holding in every world (hg recover succeeded in all 62 worlds that needed it; Borg's break-lock and check green across 119). Under the claim reading frozen before the first explore, both are precision-limit observations. Criterion-1 candidates: zero, said plainly rather than shelved — the freeze before the run is what makes that publishable. Records: spike/cohort2/RESULTS.md and the per-target RUNLOGs.
The third selection cohort ran (2026-08-22, #209): five targets, all measured, and criterion 1 stays open. Its question was the opposite end — the ordinary stateful CLI sweet spot, modern and actively maintained, with no transaction engine between the write and the disk. cargo reached a two-layer named wall (child_process_detected, then oracle_missed_operation once an owner-approved stand-in lifted it: the manifest's atomic rename is a raw syscall). (Re-measured 2026-09-16 with the released v1.4.0: the child layer no longer stands, the raw-syscall layer stands under --observe wrappers only, and under --observe syscalls cargo is judged — docs/target-classes.md, spike/dogfood/2026-09-16-cargo-v16/.) black and rustfmt each reached a reproduced checker-red FAIL in three worlds and minutes — the same in-place tear in two languages — and both closed at the novelty gate, already on their trackers (psf/black#2479 since 2021, rust-lang/rustfmt#6041 since 2024). poetry reached FAIL 2/5 whose earliest violating world is L0-only, so the frozen rule refused the claim; the checker-red world behind it is real and was reported upstream as python-poetry/poetry#11019. papis returned PASS 2/2 — one atomic renameat, no interior to crash inside. Criterion-1 candidates: zero. Records: spike/cohort3/RESULTS.md and the per-target RUNLOGs.
The fourth selection cohort ran (2026-08-23): two frozen targets, and criterion 1 has its first candidate. Its question was whether the sweet spot survives one step further out, into actively maintained tools the engine reaches only behind declared apparatus. unison never spent a define: the probe gate refused it on condition 5, two runs two seconds apart leaving different bytes in the state root, with the residue recorded unattributed after clock, pid, inode and — in the -times variants only, which the frozen argv is not — the propagated mtime were each eliminated in turn. A nondeterministic-writer wall, measured before any define existed and costing none. himalaya reached FAIL 1 of 3 worlds, oracle_verified: true, single process, the violated invariant being the declared checker rather than the built-in atomicity form, and the report carries checker_earliest (#231, ADR 0020) — which is what this cohort's frozen claim rule asks a criterion-1 candidate to be. maildir messages copy is the crate's one arm that fills the destination in place instead of staging it, so a kill after the destination is opened and before anything is written to it leaves a message file at its final path with nothing in it, which the tool then lists as an ordinary envelope. Two runs agree on every judgement-carrying field, the exhibit replays, the stock reproduction needs no apparatus at all, and the outside-the-tool recovery paths the cohort's Reporting rule demands were measured before the report went out. Reported upstream as pimalaya/himalaya#738. Fixed and closed the same day (2026-08-23): the maintainer answered "Bug fixed on master" and closed it as completed. The fix is pimalaya/io-maildir@b4e9080 — the maildir implementation is a separate crate, which is why nothing in himalaya's own history carries it — and it changes copy to land in the target tmp/ and rename into place, as store already did, so an interrupted copy leaves at worst a stray file in tmp/. That is the staging the report named as the missing one. Released as io-maildir 0.3.1 and picked up by himalaya master in d507387c; v2.1.0, the version this cohort pinned, does not carry it. Upstream added a test with the fix (tests/integration.rs). Records: spike/cohort4/himalaya-r2/RESULTS.md, spike/cohort4/probes/unison.txt.
Against §17's six conditions, scored the way the blind campaign scored them. Discovered automatically, reproducible and small hold on the record above. Novel holds on the tracker scan recorded at selection — 51 terms with positive controls green (spike/cohort4/novelty-prescan-himalaya.txt), which is evidence a known shape was looked for and not found, and all a clean scan can be. The author judges it a real bug — held by upstream rather than asserted here. The maintainer fixed it the day it was reported and closed the issue as completed, which is more than this criterion asks for: the wording is "this project's author judges the bug real, upstream confirmation is sought, not required." Sought and received. Fixed — done upstream, not by this project: pimalaya/io-maildir@b4e9080, released as 0.3.1. Kept as a regression case — the exhibit replays (spike/cohort4/himalaya-r2/run1/replay-transcript.txt), against the version that carries the bug. It has now also been replayed against a build that carries the fix, and it refuses: UNKNOWN case_no_longer_applies, exit 2, "the recording now counts 3 state-changing operation(s); the case was recorded over 2". The third operation is the rename the fix introduces, and a case names a crash point inside a recorded sequence, so the sequence it addresses no longer exists. The refusal is the promised behaviour rather than a defect (docs/contract-freeze.md §4). Measured with the cohort image untouched except for the himalaya binary, against a positive control that reproduces the committed transcript — an empty diff after normalising the minted filename, the per-run work path and the driver's own header block, which is what verify-positive-control.sh declares and performs — and a negative control showing the detector still catches the original defect: spike/cohort4/himalaya-r2/upstream-fix/. The positive-control leg runs on every pull request and every push to main (#318); the negative control and the four container runs do not, because they need the cohort image.
What that leaves is one question, and it now has a measurement under it rather than two readings of a sentence. Two of the three conditions this paragraph scored open on 2026-08-23 were closed by upstream within hours of it being written. The third turns on what "kept as a replayed regression case" requires: a case that exists and replays (true today, transcript committed), or a case wired into CI the way timew-regression wires timewarrior's — which is how this page scored the timewarrior gap when it called that one "closed as hygiene". The second reading is decidable against a bar this repository already wrote down. spike/dogfood-timew-replay.sh is that CI leg, the workflow names it "Replay across the fix: record, FAIL unpatched, PASS patched", and its leg C rejects exactly the outcome measured above: "a case_no_longer_applies here is honest but does NOT meet the v0.4 acceptance". Legs A and B hold for himalaya on the committed run; leg C is not reachable for this case. (Resolved by the ruling below, 2026-08-25.)
The asymmetry between the two findings is not about either target's quality. The timewarrior patch left the operation sequence intact, so its case survived its own fix; the himalaya patch adds an operation and its case did not. What follows from that without measuring anything further is structural: a case pins a crash point inside a recorded operation sequence, so any fix that changes that sequence orphans the case. What does not follow, and is written here as a hypothesis rather than a finding, is that such fixes are the common kind — n is two, and nothing in this record samples repairs across targets. What that implies for what a "regression case" can be — whether leg C needs a different shape for fixes that change the operation sequence, or whether an unreachable leg C disqualifies a finding — is a product question this record raises and does not answer. The adjudication is the owner's, and re-scoring the criterion is deliberately not done inside a measurement record — the same rule this repository applied when criterion 6's measured document moved underneath it.
Ruled 2026-08-25 (owner, #305): the criterion is met, and the regression-case condition gains a second accepted shape rather than a lower bar. Of the issue's three options the ruling takes A. A case orphaned by its own fix — a fix that changes the recorded operation sequence — satisfies "kept as a replayed regression case" when the exhibit replays against the pinned build carrying the bug and the defect's absence on the fixed build is demonstrated by a fresh explore under controls, which is what the upstream-fix record already measures: run 1's positive control reproduces the committed transcript (empty diff after normalising the minted filename, the per-run work path and the driver's own header lines), run 4 passes 4/4 through an instrument that relaxes exactly one assertion — the guard whose stated premise the fix falsified — while otherwise only recording evidence (checker still falsified before the run), run 5's negative control shows that relaxation leaves the original defect caught on the stock build, and run 6's functional control pins that a complete copy occurs at all. What this shape gives up is stated rather than papered over: it answers "did the defect go away", not "did this exact exhibit stop firing" — the exhibit cannot, by construction, and case_no_longer_applies is the refusal the freeze promises. Option B — leg C unchanged, such findings ineligible — was rejected for the reason the issue itself names: it makes the criterion select partly on what upstream chose to do rather than on what the search found. Option C — re-record on the fixed build — is not required by the ruling and stays available as hygiene. Nothing in code or CI moves with this ruling: the frozen checker keeps its assertions, and spike/dogfood-timew-replay.sh keeps its stricter leg C, which its own case can meet. With the ruling, criterion 1 closes on the himalaya finding (#140).
What three cohorts changed about the reading of criterion 1. Detection is not the binding constraint: from a frozen define to a reproduced checker-red verdict took minutes, twice, in two languages, on tools whose measured reach was part of their selection (#209 records black at 41,812 stars and cargo at 15,408, measured 2026-08-22). The mini-seal's machine check is recorded in eleven committed verify transcripts across the three cohorts — not one per target: the two probe walls (KeePassXC, unison) never spent a define, and poetry carries two. Until 2026-08-23 no single finding held at the same time novel and automatically discovered and provenance-clean: timewarrior lacked the first (a human read the strace to seed the window), topydo and cohorts 2 and 3 lacked the second, and the assisted cohort lacked the third (verify-assisted.sh reports its targets red — define and first artifact in the same merge). himalaya holds all three at once, and is the first to do so: the complete define was pushed before the first artifact of any exploration of that target, byte-identical at both points, which the mini-seal checks mechanically (spike/cohort4/himalaya-r2/verify-transcript.txt); the search produced the violating world from that define; and the selection-time tracker scan found nothing describing the write shape. What the transcript witnesses is the order of the pushed history — the script says so itself, and nothing in it proves what happened on a private disk first, so "no failure of the target was observed before the define" rests on the protocol with the public order as its witness, which is the strength ADR 0017 asked for and no more. What remains is therefore not the combination but the conditions a search cannot supply on its own. That sentence was written on 2026-08-23 listing three of them — an author's judgement, a fix, and a regression case that runs — and upstream closed the first two within hours by fixing the bug and saying so. One is left, and it is the one a search cannot supply and upstream cannot either: a regression case that runs. The saved case has since been replayed against a build carrying the fix and refuses (spike/cohort4/himalaya-r2/upstream-fix/). (Ruled met 2026-08-25 under the adjudicated second shape — the ruling paragraph above; #305.)
A null result was budgeted for and, in the first campaign, did not happen; campaigns 2 and 3 then spent that budget twice (the abook and khal nulls above), and the three selection cohorts spent it across twelve further targets — eleven producing no criterion-1 candidate and the twelfth, himalaya, producing the first. The novelty/confirmation/fix/replay work on the assisted findings stands beside it, unchanged by the redesign.
- Linux before macOS: retire engine risk before platform risk.
- Built-in invariant before checkers: zero-config value first; L0 is also what agent callers use.
- Dogfood after the Define contract is complete: omamori's interesting invariants need L2.
- Loop closure last: it tests the report format, which should be stable by then.
- Interposition spike fails or crash points are not deterministic → v0.1 exists to learn this in week one; kill or redesign early.
- UNKNOWN dominates in practice → measured from v0.1 onward; explicit kill criterion.
- Define burden creeps past three commands and a directory → the contract is a budget (DESIGN §12); a breach is a warning sign, not a growth opportunity.
- omamori is too hardened to yield a novel bug → the calibration target in v0.5 separates "the tool is weak" from "the target is strong".
- Name is a common word; PyPI/npm taken → accepted on 2026-08-10; distribution is a single binary.
Everything in DESIGN §15, unchanged: network faults, clock manipulation, thread scheduling, distributed systems, remote consistency, security scanning, static analysis, AI code review, formal verification, GUI, cloud, LLM verdicts, a "Passed Sideeye" badge.
Power failure / torn writes was held for consideration after 1.0 (DESIGN §21), was considered, and is declined for now: ADR 0110 gives the reasons and what would reopen it.