Sideeye — Design Document

Status: implemented and released; the five surfaces frozen at v1.0 are declared in docs/contract-freeze.md Name: Sideeye (confirmed 2026-08-10) Tagline: Sideeye doesn't believe it. Initial focus: process crash × persistent state consistency

1. Overview

Sideeye deliberately removes software from the well-behaved world it was written for, and searches for a replayable counterexample that breaks a property its developer declared must hold. It is a mean-spirited reviewer.

Ordinary tests define inputs and expected outputs. Property-based testing defines properties instead of examples and generates many inputs. Sideeye is closer to the latter, but what it varies is not primarily the input — it is the world the program runs in.

Developers do not write piles of failure scenarios ("kill here", "fail that write"). They declare invariants: if the operation said it succeeded, this must still be true after a restart. Sideeye searches for a world that breaks the invariant and, when it finds one, returns a minimal, reproducible counterexample.

The long-term surface includes process crash, persistent state, network, clock, concurrency, and partial external effects. v0 deliberately learns exactly one kind of malice: process crash against persistent state. The philosophy stays broad; the first cruelty is narrow.

2. Problem

AI coding agents write code faster than humans can understand its failure semantics. Happy paths, common error handling, and unit tests are generated in minutes; questions like what is left on disk if we die halfway? or is the state actually consistent after a restart that follows a reported success? still depend on human attention.

Outside databases and distributed systems, adversarial testing is not a normal part of development. Ordinary CLIs, developer tools, daemons, and desktop applications work fine for months, then fail at the first coincidence of a crash, a partial state update, and a restart.

The problem Sideeye attacks is not "not enough tests." It is that failure worlds the developer never imagined cannot be verified by enumerating them as test cases in advance.

3. Product Thesis

Sideeye is a deterministic adversarial gate for the coding loop.

The bet has two halves:

Asking developers for invariants — and letting Sideeye invent the failures and shrink the counterexamples — is what makes adversarial testing viable for ordinary software development.
In a loop where more and more code is written by LLMs, the skeptic must not be one. A gate only works if its verdicts are deterministic, reproducible, and machine-consumable.

Sideeye does not aim to invent new fault-injection primitives. Property-based testing, shrinking, fault injection, crash-consistency testing, Jepsen-style adversarial verification, and deterministic simulation all have long lineages. Sideeye's bet is on bringing those ideas to ordinary stateful software as an invariant-first developer experience — cheap enough to sit inside the loop:

write → test → sideeye → review → ship

4. Core Principles

4.1 Ask for invariants, invent the failures

The central thing a user writes is not how to break the software but what must not break.

Invariants come in two shapes, and the distinction matters:

Always-invariants (crash atomicity). For every crash point: after restart, the persistent state is either the complete old state or the complete new state, and diagnostic commands must agree with reality. These hold unconditionally across every explored world. They are the workhorse of v0.
Post-success invariants (durability). In worlds where the operation reported success before dying, the new state must survive restart. Sideeye records what the program claimed on stdout up to the moment of the kill, and holds the program to its own words.

A conditional invariant of the form "if it succeeded, then…" is vacuously true in most crash worlds — the killed operation never reports success. Sideeye's defaults are therefore built on always-invariants; post-success invariants are the sharper, optional layer.

4.2 Break worlds, not just inputs

Sideeye does not primarily generate strange inputs. It takes a normal input and breaks the environment's promises partway through — the process dies. The question is not what if the input was bad but what if the world was not convenient.

4.3 Counterexamples, not warnings

The central artifact is not a warning or a risk score. It is a reproducible counterexample.

Bad output:

Possible inconsistency during state replacement.

Desired output:

For an execution where rotate-key reported success, killing the process at a specific point leaves no valid key after restart while doctor still reports healthy. Reproduce in 3 steps.

"This might be dangerous" loses to "this actually broke, here is how."

4.4 Shrink aggressively

The first failure scenario Sideeye finds may be complicated. It is never handed to the developer as-is: unnecessary conditions are removed until only what is required to break the invariant remains. Failure reduction matters as much as failure discovery.

In v0 the search space is (one operation × crash points), so shrinking degenerates into choosing the earliest, simplest failing crash point. That is a benefit of the scope, not a limitation of the idea; shrinking earns its name once operation sequences arrive.

4.5 Never overclaim

Sideeye is suspicious of itself too. "Found no counterexample" and "proved it safe" are different sentences.

Results distinguish, at minimum:

PASS — no invariant violation found within the declared exploration.
FAIL — a reproducible invariant violation was found.
UNKNOWN — the result could not be judged correctly.
NOT TESTED — this area was not part of the verification.

PASS never means "crash-safe" as an unbounded claim. This vocabulary is enforced by the exit-code contract (§13), not by prose.

4.6 Failure is part of product behavior

Behavior under failure is product behavior, exactly like behavior under success. "Anything goes, it was an error path" is not accepted. In particular, three things are treated as explicit contracts: the state after a reported success, the state observed after a restart, and the state a diagnostic command reports.

4.7 The skeptic is not an LLM

Everything that affects a verdict — exploration, violation detection, shrinking, replay — is deterministic and does not involve a language model. LLMs are welcome at the edges: proposing invariants on the way in, explaining reports on the way out. They never decide PASS or FAIL.

In a development loop where the code author is increasingly an LLM, the reviewer's value is precisely that it is not one.

5. Product Personality

Sideeye is not a clever AI reviewer. It does not read your code and generate ten concerns.

Its personality is the suspicious senior reviewer with unreasonably good instincts. "The tests pass" gets "and if it dies halfway?" "We retry" gets "and if the previous attempt half-succeeded?" "Doctor says healthy" gets "and if doctor is wrong?"

The personality lives in the product's stance, not in the output. Findings are calm, unambiguous, and machine-usable. Sideeye doesn't believe it is an attitude, not a tone of voice.

6. Why Sideeye, Why Now

As implementation gets cheaper, sustained human suspicion becomes relatively more expensive. With more code, faster change, and multiple agents editing in parallel, no human reviews every failure path in their head.

What the loop needs is not only a faster generator. It needs something that doubts at the speed the generator writes.

Sideeye is not an AI-only tool. But it is designed, from v0, to be callable by agents: non-interactive, machine-readable, exit-code honest. This section is not background color — it is where the output-contract requirements in §13 come from.

7. What Makes Sideeye Different

Three differences are the point; none of them is a new primitive.

Ordinary software. Adversarial testing for CLIs, local developer tools, and small daemons — not only for databases and distributed systems.
Invariants first. Users write "if the operation succeeded, this property holds after restart" — never "SIGKILL here, then fail this write."
Replayable counterexamples as the deliverable. Evidence a developer (or an agent) can paste into an issue, a regression test, and a fix verification, unchanged.

Whether Sideeye deserves to exist is decided by whether these three can be made easy for ordinary developers.

8. Design Lineage

Sideeye comes from a lineage of projects (omamori, llm-key-ring) whose recurring theme is refusing to trust reported success: measure before believing, falsify a new guard once before trusting it, never treat an unverified state as success. Sideeye turns that habit of suspicion itself into a standalone tool.

Someone else building on the same ideas would likely make a fault-injection framework or a chaos-testing tool. Sideeye's identity is not the number of things it can break. It is the stubbornness: refuse vague success, produce an actual counterexample, shrink it, bring it home.

9. v0 Scope

v0 answers one question:

If the process dies partway through a stateful operation, do the developer's declared invariants still hold after restart?

Typical targets: config updates, key rotation, state migration, index updates, cache metadata, audit state, anything around a local database.

The crash model, precisely

v0's crash is a process crash (SIGKILL-equivalent): the OS survives. Every write the process completed is durable; nothing is lost or reordered. Power loss and kernel panics — where unsynced data vanishes and write order gets rearranged — are outside this model, and a PASS says nothing about them.

This limitation is a focus, not a weakness. The bugs that surface under the process-crash model are pure logical-ordering bugs — delete-then-create, missing temp+rename, print-success-then-write — exactly the class ordinary software is dense with. It also keeps the engine simple: no filesystem simulation is required.

Known constraints (declared, not hidden)

Targets must keep persistent state in files or directories Sideeye can snapshot and restore. State in system keychains or remote services is out of scope. Symlinks inside the state directory are snapshotted and restored as links.
The recording run and every world start from the state Sideeye rebuilt from its snapshot, not the one the setup left (#815, ADR 0118). The rebuild empties the state directory and writes the entries back in name order with their bytes, link targets and permission bits (a directory's owner always keeps rwx); it carries no owner, hard link, set-id or sticky bit, timestamp, extended attribute, ACL or file flag, and a state directory the setup removed comes back as an empty one. Until #815 only the worlds started there, so a target that read any of that — or listed its state, which tmpfs, btrfs and XFS were measured listing in another order once rebuilt than as the setup wrote it — could do one thing in the recording run and another in every world while the crash points stayed numbered by the recording. A target that depends on the setup's own copy now meets the rebuilt one from its recording on. The recording and the worlds start from the same rebuild, not from byte-for-byte the same directory: on tmpfs they list it alike (measured, acceptance check 27), and a filesystem that places a new entry by where earlier ones were removed can list two rebuilds differently. A state the engine cannot rebuild is refused before the operation runs. preflight rebuilds before its one run, and --twice before each of its two, so preflight starts where explore's recording does. Each command rebuilds once more than before — preflight from none to one — at the cost of one world's rebuild.
A world is judged only when it reached its crash point through the operations the recording numbered (#815, ADR 0119). Each crash point is a number in the recording's count, so a world killed at number k is held, for every operation before k, to the recording's: of the same class, and at the same path wherever the world names a path the recording or its two snapshots spell, or the recording named one the state started with — a rename's or a link's two paths each. A world that is not is refused kill_did_not_land, naming the first operation out of place in the world and in the recording, and its state is not judged. What is not compared: the bytes written, an operation that is not a crash point, an fsync, a mkdir on a directory that was there from the start, before anything in the run was removed or renamed, and a name the recording never spelled where the recording's own name was not one the state started with — a pid, a random suffix or a clock in a scratch name. So for a target whose order varies between runs, a verdict comes out only when the recording and every world before the baseline drew the same order, and it is a verdict about that order: on #815's shape — two files written in a Python set's order under a random PYTHONHASHSEED, with four crash worlds, the three after the first compared — 12 of 120 explorations reached one (4 PASS where the recording wrote beta first, 8 FAIL where it wrote alpha first, each with every compared world in the recording's order), and the other 108 were refused; before this, 7 of 30 PASSed, 6 of them with a compared world writing the other order and 4 having recorded alpha first, the order the checker fails in (spike/dogfood/2026-10-11-order-815/). Pinning the source of the order — PYTHONHASHSEED in the define's environment, declared as apparatus — makes every run reach the same verdict.
Under the default mode and --observe syscalls, Sideeye targets dynamically linked executables. macOS binaries with hardened runtime + library validation, and statically linked Linux binaries, cannot be observed by a preloaded shim; Sideeye refuses them (no_shim_marker, UNKNOWN) instead of pretending. A statically linked Linux target is observed under --observe supervised (Linux 5.19 or later, with a cgroup v2 the engine can create cgroups in), which counts its calls from outside the process with no shim loaded (ADR 0089); a hardened macOS binary has no such mode.
A target that creates other processes is explorable when an oracle can account for all of them. Until contract v15 the rule was that no process other than the subject touched the state directory; since then a run in which no two processes' state-directory operations interleave and every writing child is reaped by its parent is explored and judged, with the children's operations numbered in the run's own sequence and killed like any other crash point (ADR 0053). What still refuses is a run whose writers overlap, a writer nothing waited for, and a writer the shim never loaded. --observe syscalls refused the whole class until ADR 0054 gave that mode an oracle on the run its trace comes from; it is judged there now like any other mode. It requires --oracle, on Linux: macOS's --oracle-fs-usage cannot account for other processes, so a process boundary the shim saw is refused under it (boundary_without_oracle, ADR 0031). A single-threaded target that execs over itself is judged under --oracle on Linux when the carried operation count survives the image change (since contract v10, ADR 0018), and refused when the chain breaks. On macOS, where the chain holds, the image change is refused as a boundary (boundary_without_oracle) under --oracle-fs-usage as with no oracle at all, and a target that writes on both sides of it is refused before that as two writing threads, because the thread that survives an exec there reports a new id (#571); where the chain breaks, it is refused as a broken chain there too. A process boundary or an image change in a run given no oracle is refused. A process that leaves the process group (setsid, setpgid) is judged where the engine held the run in a cgroup of its own (contract v17, ADR 0065): on Linux, where the engine can move processes within its own cgroup — a writable cgroup v2 delegated to its user, or root over a writable one — and every image the shim was loaded into said it was inside the run's cgroup before doing anything else, that process is stopped at the crash point and at cleanup with the rest of the run and judged by the rules any other child is. A world holding a record from after its crash point is refused, and so is a run with a process seen outside its cgroup or alive after the cgroup was stopped. What containment cannot see is a process that leaves both the cgroup and the process group where the shim does not ask — at no boundary, in no image the shim is loaded into — touches nothing in the state while its world runs, and writes afterwards, where a later world can take it in (ADR 0065). Everywhere else the detach is refused child_process_detected, with the environment as the next step on Linux and, on macOS, a step naming a Linux run where the engine can make cgroups. A target that creates threads is judged when every two of its state-directory writes within one process are ordered by a thread creation or a join the shim recorded, however many threads there are, and refused when any two are not (contract v16 admitted one writing thread per process, ADR 0055; contract v18 admits the ordered ones, ADR 0067) — decided from the trace, since every record names its thread and a thread's first record names its creator and where in the creator's own records the creation fell, so an explored world and preflight's second run answer it for themselves rather than inherit it, and it needs no oracle, because a thread's writes reach the shim that shares its process. What the shim sees of the thread API is pthread_create, pthread_join and pthread_detach; a thread's start is recorded by the thread itself, before its routine runs, and a join is resolved before the real call, while the handle is still the collected thread's (glibc reuses it the moment the join returns). Two threads a queue, a mutex or a condition variable orders are ordered by nothing the shim sees, and refuse — beets' pipeline stages are the measured case. Under --oracle-fs-usage the same rule decides it (ADR 0060). That oracle attributes a line to a thread id and knows no process for it (ADR 0031), so the map from thread to process comes from the side that has it: the trace names both on every record, and the subject's writing thread ids are handed to that reader, which counts their lines as the subject's. A single-process run that neither creates a child nor replaces its own image, and whose state-directory writes come from one thread the shim recorded — the main thread or a worker — is explored and judged there too, and a second writing thread of that process is judged or refused by the same order rule above, from the trace, as on any other platform. Whether that list holds a given writer turns on which of two records the shim missed, and the bullet below says which is which. A single-process target run with no oracle at all reaches PASS only under --allow-unverified, and the report says the weaker claim out loud.
Two records can go missing, and they are not the same miss (#543). If the shim missed the creation — a thread reached through a raw clone, which writes no pthread_create record — it still records every operation that thread performs through libc, because the wrappers are process-wide and each record names the thread that made it. Such a run is judged on those operations, under the same order rule as any other thread — a thread whose creation the shim missed has no creator to inherit an order from, so its writes are ordered with nobody's and a second writer refuses, whether the shim saw it created or not — and the account says that a thread the shim never saw created wrote in the judged directory. If the shim missed the operations — a writer going straight to syscalls — it has nothing to attribute, and the run is refused, naming the writer as an id rather than as a process: multiple_threads_detected where the shim recorded threads being created and no process boundary, child_touched_state_dir where it recorded neither, which is the raw-fork shape (#405). Which of the two the run can establish is decided from what the shim saw, because the oracle cannot see it at all. A writer that is both is the second case. Which case a target falls in can depend on the mode: --observe syscalls records the operations of a writer that goes straight to syscalls (#542), so a thread that misses both ways under the default mode is only a creation-miss under that one, and the one-writer rule decides the run as it would for any recorded thread. Two calls are exempt and stay an operations-miss in either mode — copy_file_range and pwritev2, for the reason the bullet on that mode gives below. A Go runtime's threads are the case to expect here. The consequence of the first is that the count of threads the shim saw is a floor, and the two witnesses are never compared on it: strace reads CLONE_THREAD off the line and takes the thread for the subject's, the shim has no record to offer, and nothing reconciles the difference. A thread that is neither recorded nor writes anything leaves the trace nothing at all, and no account can name it.
A process that creates more than sixty-four threads is refused (#543), counted over the whole run rather than by how many are alive at once. The shim's slot table holds that many and never reclaims a thread's: nothing in it observes a thread ending, so a target that starts a worker, joins it, and starts another has spent two slots. The sixty-fifth taker gets a shared reserve slot, the run records that once in front of that thread's first operation, and the engine refuses on the record. A pool that recycles work across more than sixty-four spawns is therefore refused even though it never has two threads running together — where its workers enter the shim's wrappers at all: a slot is claimed by a thread's first interposed call, and the start record a thread writes since v18 (ADR 0067) is written from its own stack and claims none, so seventy threads that write nothing and open nothing are judged. On Linux, a child started through vfork gives back the slot it claimed when it execs through execve, execv or execvp (#821, ADR 0127): it runs in its parent's memory and claims a slot there under its own thread id, so before this a single thread that started seventy children that way was refused like a pool of seventy threads. While it runs, the child holds that slot, so a process already at sixty-four threads is refused when it starts one. A vfork child that leaves another way — one whose every exec fails and that then exits, one that execs through execvpe, execl or fexecve, which the shim does not interpose, one that execs before the shim's constructor has run, one whose signal handler makes an interposed call between the give-back and the exec — keeps its slot while it lives, and once it has been reaped the slot is taken back when the table is next full (#901): a slot whose claimer kill reports gone is reclaimed before the reserve is handed out. A target that starts more than sixty-four such children before it reaps any is refused as before. macOS was not measured. This is a capacity limit rather than a third way of missing a thread — a target under it is unaffected, and a target over it is refused rather than mis-attributed.
Where a write is counted is a choice, and there are three. The default counts the write family at the interposed libc entry points, with buffered stdio observed at flush granularity (ADR 0005) — so a write libc issues from inside itself, an fwrite past the buffer above all, is seen by the oracle and not by the shim, and the run is refused rather than misjudged. --observe syscalls (Linux) counts at the kernel boundary instead, through a seccomp filter and a SIGSYS handler inside the target's own process, which reaches both that case and a raw syscall(SYS_write, …) (contract v14, ADR 0052). Since #542 that mode's trap set is every operation Sideeye counts as a crash point — open, write, rename, unlink, fsync, truncate, mkdir, rmdir, link, symlink, in each spelling the architecture has — so a target that issues them without passing through libc is judged rather than refused, which is what a Go runtime is (ADR 0059). Two calls stay outside: copy_file_range and pwritev2, whose six arguments leave the filter no free register for the re-issue marker; a raw one of those is seen by the oracle alone, which refuses. What --observe syscalls leaves uncounted is those two, a process running in a syscall ABI the filter cannot read, and any image the shim is not loaded into. The filter admits an open or openat only when its flags say the call can change something, which is what lets an exec'd image's loader survive it (openat2 carries its flags in a struct that a BPF program cannot read, so it is trapped unconditionally and the handler re-reads them; glibc's loader uses openat), and the shim interposes the four libc entry points that set the SIGSYS disposition or mask so that a target cannot take the handler away — accepting each request and answering success without applying the part that would. Under an oracle the second mode watches the run whose trace is judged, like every other mode: a trapped write reaches strace twice — refused, then re-issued — and the reader retracts the refused entry when it reads the SIGSYS that refused it. So the claim is oracle_verified, and the report's oracle line says how the capture was read. It reported oracle_verified_across_runs until 2026-09-08 (ADR 0054), while its oracle watched a separate untrapped run and the two accounts were of two executions; the contract number did not move for either the addition or the withdrawal, so both sit inside v15. The third, --observe supervised (Linux), counts from outside the target with no shim loaded at all — the mode for a statically linked target (ADR 0089), and the answer to #217, closed on 2026-09-27; docs/cli.md's "How supervised mode counts, and what it refuses" names what it still refuses.
Some calls put bytes in a file and are recorded by neither witness, and the per-path reconciliation catches them only where no record able to put those bytes there names the file (#814, ADR 0121). On Linux, a reflink clone (ioctl(FICLONE) / FICLONERANGE — GNU cp's default on btrfs and XFS) is not interposed by the shim and was read-only to the oracle's syscall table, and a write submitted through io_uring reaches neither: strace sees io_uring_enter on the ring's descriptor, outside the judged directory. Until #814 a file truncated by a recorded open(O_TRUNC) and refilled through one of those passed with a verified oracle, and the world holding the truncated file was never explored — measured on btrfs, XFS and io_uring, PASS 4/4 each, where the same cp on overlayfs falls back to copy_file_range and FAILs. A file the run leaves holding bytes, outside a path the define declares scratch, is now accounted for only by a recorded write, the destination of a rename or a link, the source of a link whose other name was written after it, or a truncate whose result is the old bytes cut short or padded with zeros (docs/report-schema.md, state_changed_unaccounted), so those runs are refused. What the reconciliation cannot reach, being per path and reading only the two snapshots, is such a call on a file one of those records also names — a recorded write beside it, a rename or link that put the file there first, or a truncate whose result the final bytes happen to match (ftruncate(fd, 0) and then the old bytes' first part, through io_uring) — or on a scratch path, which a checker can still read: the file is accounted for, and the window in front of the unrecorded call is not explored. The same reconciliation now refuses, in the default mode, a run whose only record of a file's new bytes was lost inside libc — a stream left for exit to flush, an fwrite past the buffer (ADR 0005) — where the open used to account for it; on Linux --observe syscalls counts those writes. Under --oracle a clone into a state file is refused outright on the recording — unsupported_syscall_observed, ioctl(FICLONE), from the subject or a child (a child the v15 rule does not admit is refused first, as that), whatever else names the file — which narrows that gap for clones (explored worlds read no oracle); a whole-file clone that failed before remapping anything (EOPNOTSUPP, EXDEV and the like) is read as a read, since cp then falls back to copy_file_range, and any other failed clone is not, since btrfs and XFS can fail after remapping part of the file. io_uring is not refused, by the owner's call (2026-10-11): Node 22.23.3 sets up three rings at start, with UV_USE_IO_URING=0 too, and its file writes still reach the kernel as plain writes (measured with strace 6.1 outside Docker's default seccomp, which refuses io_uring_setup), so refusing a ring would refuse every Node 22 target on a host that allows io_uring and buy nothing for the targets measured. Other calls that change a file's bytes through ioctl — BTRFS_IOC_ENCODED_WRITE, XFS_IOC_EXCHANGE_RANGE, and FS_IOC_ZERO_RANGE and FS_IOC_UNRESVSP/FS_IOC_UNRESVSP64, which reach fallocate's zero-range and punch-hole on every filesystem since Linux 5.5 while the subject's own fallocate with those modes is refused — are in the reconciliation's position, not refused by the oracle.
A target that execs an image the shim cannot be loaded into must not use --observe syscalls. That mode installs a seccomp filter, and a filter is inherited across exec and cannot be replaced, while the SIGSYS handler that makes it survivable is reset by exec. So a statically linked helper, or any image the preload does not reach, dies at its first state-changing call — measured: exit 0 under the default mode, killed by SIGSYS under this one. It does at least get to start: the filter admits an open or openat whose flags say it cannot change anything, so the dynamic loader opening its libraries is not what kills it — a loader that used openat2 would still die, since those flags sit in a struct the filter cannot read. Every other constraint in this list makes Sideeye refuse — save a child that dies of a blocked SIGSYS, the next one's case; this one changes what the target does, which is why it is the one to check before reaching for the flag — this one and the shim's own footprint, below. A child the shim is loaded into installs its own handler and is unaffected, in either mode: the handler goes up whether or not that process installed the filter, precisely because it may have inherited one — from the shim's constructor on. A call another library's constructor makes before that, in such an image, meets the inherited filter with no handler yet and ends the process (#753, below). The exception is a child that has not reached an image of its own: glibc's posix_spawn runs its file actions in the new process before the exec, with every signal blocked, so a file action that opens for writing is trapped there and kills the child — measured: signal 31 on glibc 2.36, under a preflight that was accepted; that file action opened outside the state directory, so a run judged without a dead child's state-directory work was not measured.
Under --observe syscalls a process whose SIGSYS is blocked or reset dies at its first state-changing call, and a target can notice the shim guarding it. It interposes sigaction, signal, sigprocmask and pthread_sigmask: a request naming SIGSYS is accepted and answered successfully, and the part that would replace the handler or block the signal is not applied. sigaction also takes SIGSYS out of the sa_mask of a request for any other signal and applies the rest: the kernel blocks a handler's mask while the handler runs, so a handler installed with every signal there — libuv installs its handlers that way, which is how node 20 was refused recording_run_failed — would hold SIGSYS blocked and die at its first trapped call. A target that queries such a handler reads its mask back without SIGSYS. Only that mode declines anything. The four symbols are exported in every mode — a symbol cannot be exported conditionally — so under the default mode a target's calls to them are forwarded to the C library's, and nothing is refused, reported differently or left unapplied. What the default mode does get is the forward itself: one load and one indirect call more than a direct one. Without this a Go target — which installs its own SIGSYS handler through libc, and touches the signal mask through it — takes the mode with it, because a trap on a thread with SIGSYS blocked ends the process rather than reaching any handler. What does not reach those four symbols while their guards are up is outside this and still dies — a raw rt_sigaction or rt_sigprocmask (a statically linked Go binary is the case to expect), the C library's own internal calls and its other signal functions, a mask held for the length of one call, a call made before the shim's constructor arms the guards, a child that kept the shim but lost Sideeye's environment — and docs/report-schema.md item (4) names the ways known. Trapping those calls in the filter instead was measured and not taken, because it kills targets that run today (ADR 0063).
On Linux, a call made before the shim's own constructor has run reaches the real function, and is not counted (#753, ADR 0115). Another shared library's constructor can run first — the loader orders constructors by dependency, and nothing a library can declare puts its own ahead of an unrelated one's — and until #753 a call interposed from one answered -1: C++'s std::thread aborted on it and a std::ifstream failed, in targets that ran by hand. Loading the shim no longer changes what such a call returns, with the exceptions ADR 0115 names: under --observe syscalls, in an image that mode exec'd into, the filter is already in place and its handler is not, so such a call that traps ends the process (recording_run_failed, the limit above); a thread such a constructor started with SIGSYS blocked dies at its first trap once the filter reaches it (the guards' gap (e) in docs/report-schema.md); and before the constructor each call looks its function up with dlsym, which a signal handler, or a thread waiting on a dlopen in progress, can meet the loader's lock in. What the call does is not numbered, so one that reaches the judged state — a write-capable call on a path inside it, an fopen for writing included, or through a descriptor that points there, inherited from an earlier image included — refuses the run (unresolvable_path, naming the path): no crash point can be placed in front of it, and a later recorded write to the same path would otherwise read as having accounted for it. The mark survives an exec made before the constructor through execve, execv or execvp — the entry points the shim interposes — in the environment; it does not survive the execl family, execvpe, fexecve or execveat, which reach the kernel from inside the C library (the escape ADR 0018 names for the operation count), nor a process that ends before any constructor of the shim's has run in it — a child a constructor forks and that exits there. Both are the gap a process the shim never saw already is; with an oracle, the oracle sees the write. A thread such a constructor starts is under --observe syscalls' filter too (it is installed for every thread). macOS calls the originals directly and is unaffected.
The shim's footprint on a target thread is bounded, as measured on Linux. It takes less than 1 KiB of a thread's thread-local storage, and at most 5 KiB of its stack for an interposed call — plus, under --observe syscalls, the kernel's own signal frame (AT_MINSIGSTKSZ) for each trapped operation. Since #542 the handler carries SA_ONSTACK, so on a thread that has an alternate signal stack — every thread a Go runtime makes — that frame and the handler's own come off that stack rather than the thread's; the measurement below is of the thread stack, and the alternate one is not measured. A thread that has less than that left at an interposed call can fail under observation where it would not without it. The 5 KiB is a Debug build's, which is what zig build makes; the release build measured under 1 KiB. Acceptance measures all three on Linux on every run (spike/check-shim-footprint.sh, #555), the stack over open, write, close, rename, unlink, mkdir, rmdir and a failed exec; the macOS dylib is not measured. Through 1.3.0 the shim carried 256 KiB of TLS, which stopped a loaded target from starting a thread with a smaller stack, and up to twenty kilobytes of stack per call.
Writes must repeat byte-for-byte across two clean runs from the same state. The un-killed baseline world is a fresh execution compared against the recorded final, and every crash world is judged against that same recording; a mismatch that appears without a crash cannot be attributed to one, so a target whose output carries the clock, a random id or a cache keyed on an inode the restore moved is refused there (baseline_violates_invariant, naming the path and what was observed of it) rather than judged. The engine does not guess which of those it was. What it does say is what the bytes show (#688): where the two runs' bytes first differ, how long the differing stretch is in each run and what kind of bytes it holds — decimal digits, hex digits, printable text, or how many bytes fall outside printable text — and never the bytes themselves, because the refusal reaches the JSON report and the MCP answer and a stretch two runs disagree on is where a per-run token sits. preflight --twice says the same and quotes both stretches, in its own output only. preflight --twice measures the same property before a define is written (#199). A path the define declares in scratch is outside this constraint: neither the judgement nor --twice compares it, and the report says so (ADR 0043).
A state file written through a shared memory mapping is refused, not judged (#689, ADR 0098). A store into a MAP_SHARED mapping changes the file with no call behind it, so no crash point can be placed between two stores, and a PASS would claim states nobody built. A run in which a file under the state directory can be written that way — mapped writable, or given PROT_WRITE by mprotect after it was mapped read-only — is refused unsupported_syscall_observed, unless something else refuses it first (a child the v15 rule does not admit is refused as that): on Linux the strace oracle sees it from the subject and from every child, and on macOS the shim does, with or without an oracle (contract v19). A read-only mapping is remembered only where it can become writable: on macOS the shim asks the descriptor how it was opened; on Linux the oracle remembers the state files this run opened for writing, matched by name, and every state file mapped shared once the run has renamed or linked anything (a name strace prints is the one the file had at that call). So a read-only file mapped and unmapped, in a run that renames nothing, leaves nothing a later mprotect in that run can be refused against. A read-only shared mapping is judged like any read — bbolt maps its file that way and writes with pwrite. On macOS this newly refuses what Linux already did: an SQLite database in WAL mode (its -shm file is mapped writable) and LMDB (its lock.mdb), when either is in the state. What stays unseen: on macOS, the Mach calls that map or protect memory without mmap or mprotect (vm_protect, mach_vm_protect, vm_remap, mach_vm_remap, mach_vm_map of a memory entry) and a program importing the syscall stubs __mmap or __mprotect by name; on both, a file mapped writable outside the state and renamed into it; and on Linux a run with no oracle, which reaches PASS only under --allow-unverified. Until v19 macOS judged these runs: a target that stored into two pages of a writable shared mapping passed 3/3 with a checker that rejects one page new and the other old.
Exploration runs one world at a time, and each world restores and re-reads the whole state directory, so a run's cost grows with the number of files under --state times the number of crash points; the three ways to lower it — choosing crash points, differential restore, parallel worlds — are post-1.0 work whose prerequisites ADR 0042 names. Measured on one laptop under load, medians of three runs, no oracle, no checker, no forking target: under 0.2 s per world plus 1.4 to 2.2 ms per file per world (20 padding files 0.17 s, 2,000 files 3.7 s, 20,000 files 28 s; the clock includes the recording run, so each figure is an upper bound on one world), and bytes are secondary (five times the bytes at 2,000 files cost 8% more) (spike/explore-cost/; held by review, ADR 0039). The crash-point axis was measured separately and the product is an over-estimate (spike/explore-cost/scale/, ADR 0081 and 0082): holding the state tree fixed and varying crash points from 12 to 1002, the state tree's multiplier falls from 13.6× to 7.2× as the world count rises, because its fixed per-run cost is amortised over more worlds, and total wall grows as the world count to the 1.10 over the measured range (the exponent is not constant: 1.33 over the top decade alone). On that record a thousand crash points cost 4.2 s on a one-file state and 30 s on a 20 MB one, and peak memory rose 5.1% over 77× the worlds — it is dominated by the state tree, which costs 6.0× at a fixed world count, but it is not flat. The first of the three lowerings above — choosing crash points — was measured and found unwarranted over that envelope (ADR 0082). The measurement is a container on one laptop, not a CI runner, and the clause it was judged against is an absolute one; the form written above, longer than a project's other checks, is a ratio that record does not compute.
On macOS, a framework Python's bin/python3 is a launcher, not the interpreter (#703). Built from CPython's Mac/Tools/pythonw.c, it replaces itself with the framework's interpreter through posix_spawn with POSIX_SPAWN_SETEXEC; the shim's posix_spawn records a spawn only when the call returns, which that one never does, so the new image announces itself with no exec record and the chain of observation breaks. The refusal stays — a hand-off nothing recorded is an image change no witness accounted for — and it names the way past: the interpreter the launcher hands the run to, which is observed directly, and in a virtual environment the __PYVENV_LAUNCHER__ value the launcher would have passed. Both are read from the layout pythonw.c relies on, not from a list of distributions; a layout that does not hold keeps the refusal's ordinary step.
v0 runs natively on macOS and Linux. (On macOS, every language is forced through libSystem, which makes userspace interposition a single mechanism covering Rust, Go, Python, and friends; the same approach covers dynamically linked Linux binaries.)

The one named exception: a directory renamed in from outside

A directory a recorded rename moved into the judged tree from outside it is attributed to that one record, because its source was never snapshotted — so a later unrecorded write inside that subtree can still ride a PASS. The JSON report counts the paths that attribution covered (paths_attributed_to_rename) and the text report states how many on its atomicity line when there are any; a run counting zero has no such gap. The README names this in the sentence that makes the promise it qualifies, "never a silent PASS", rather than here alone (ADR 0032).

Not in v0

Network failure, clock manipulation, multi-process races, distributed systems, and partial remote effects are not goals for v0. They remain future possibilities — not before the first product value is proven.

10. Primary Users

Sideeye has two users per run, and they are usually different parties:

The caller is often not a human: a CI job or a coding agent invoking sideeye non-interactively and branching on its exit code.
The reader is a human — or an agent acting for one — who receives the counterexample and decides what to do.

The first human audience is OSS and developer-tool authors who:

build stateful CLIs or local applications,
have decent normal-path tests,
care about crash consistency but will never write tests for every failure point,
keep state in copyable files and can check it after a restart,
want reproductions, not warnings.

The first dogfood target is omamori. Not a product requirement — but its author understands its internal state and failure semantics, and it already carries many hand-written adversarial tests, so it sets a deliberately high bar for whether Sideeye can beat existing human tests. (See §17 and §18 for the calibration risk this creates.)

11. Core User Story

As a developer, I declare the property a stateful operation must keep, and let Sideeye explore the worlds where that operation crashed halfway.

When Sideeye finds a violation, I receive the shortest possible reproduction and observation — not an essay about risk.

After I fix it, I replay the same counterexample and watch it stop reproducing.

When the developer in this story is a coding agent, nothing changes except that nobody is watching: the report has to be actionable without a human translating it.

12. Conceptual UX

Three verbs, nothing else: Define. Explore. Explain.

Define — the target operation and the property that must hold after it (or after a restart).
Explore — Sideeye searches crash worlds of that operation. Where to crash and what state survives are Sideeye's responsibility, not the user's.
Explain — on a violation, Sideeye shrinks the counterexample and returns the minimal conditions, the expected state, the observed state, and the measured reproducibility.

The v0 contract (a budget, not an implementation spec)

What a user writes in v0 is three commands and one directory:

# sideeye.toml
[world]
state = "./state"                 # the directory Sideeye snapshots and restores

[define]
setup     = "mytool init"         # produce the initial state
operation = "mytool rotate-key"   # what Sideeye kills partway through
check     = "./check.sh"          # runs after crash + restart, in a fresh process
                                  # exit 0 = invariant holds

If Define ever needs more than this, that is movement toward the kill criteria in §18, and we should notice.

Noticed three times, deliberately, and recorded each time: marker joined with the L1 layer (ADR 0008), expected_status joined for targets whose success convention is a non-zero exit — git-style tools were unjudgeable without it (ADR 0014) — and the argv form joined for the argument a space-split string cannot spell, after the #84 sweep measured the script-file escape hatch being refused structurally for exactly the targets that needed it (ADR 0019). The third is not a key at all but a second value shape for the existing commands; the sentence above covers shapes as much as keys, and a sixth key — or a third shape — should face it again.

Faced a fourth time, for cwd. §18 has no row about Define growing — the sentence above says growth is movement toward those criteria, and the two it moves toward are row 6, setup too heavy for ordinary software, and row 2, defining invariants costing about what ordinary failure tests cost. Against those two the answer is that this key removes setup rather than adding it: what it replaces is a launcher script the author had to write and commit beside the define, which is strictly more setup than a line in the file. It also says nothing about the target's behaviour, which is the shape that would make a define grow toward row 2 — a declaration that has to spell out what the tool does is a judge that has stopped being able to watch. It says where the engine starts it — a choice every caller already had except one. The MCP server is handed a config path and starts the engine itself, so the directory was settable by everyone but the caller this tool exists to serve, and the flag added alongside the key only restates at the CLI what cd already did. The gap's measured shape is in this repository: spike/cohort3/*/ops/explore.sh are committed launchers whose content is a cd and some environment, written because the define could not carry them. What would count as Define growing, and stays refused, is a key describing the target's behaviour rather than its invocation. The environment is the nearest sibling and was deliberately left out: SIDEEYE_MCP_CHILD_ENV and the CLI's inheritance already give every caller a way to set it, so it fails the "no other way to say it" test cwd passes.

Faced a fifth time, for apparatus (ADR 0041) — and the paragraph above is the test it had to pass, because the environment is exactly what it names. It passes for a different reason than cwd: it is not a way to set the environment, which stays where the paragraph above left it, in the launcher, in setup and in the allowlist. It is the only way to say, in the artifact the report carries and a reviewer reads, that this define assumed these devices — a faked clock through a global preload, a sitecustomize on PYTHONPATH, a stand-in compiler — and to have the engine hold the run to that claim before anything is recorded. The measured gap is again in this repository: every cohort that pinned a clock did it in ops/explore.sh and setup.sh, and the committed toml said, in a comment, that the "apparatus reaches the operation child from the launcher's environment"; the same toml explored without the launcher is a different experiment that refuses at the baseline without naming what was missing. Against §18's rows it removes setup rather than adding it — the launcher script is what a line in the file replaces — and it describes the invocation's surroundings, not the target's behaviour. The engine applies nothing; the key is a declaration it verifies, and the two devices that cannot be declared (a seccomp profile is the container's; a pid pin applied inside the operation's own command line is not the engine's to see) are carried as note: and reported as unchecked rather than silently assumed.

Faced a sixth time, for scratch (ADR 0043) — and this one reverses a ruling rather than extending one. Four times a path nobody depends on decided a verdict (git's COMMIT_EDITMSG, buku's journal, Borg's relocated cache, Mercurial's dirstate), and the recipe each time was a checker carrying the real integrity claim; the recipe is right about knowledge and cannot touch the verdict, because a checker adds and L0 fires whether or not one exists. The key passes the test above for a reason neither cwd nor apparatus needed: there is no other way to say it. A checker cannot subtract from L0; --state fixes the range of the question at directory granularity and cannot be narrowed around a file inside the repository it judges; moving the path out needs the target's cooperation; not tested is the engine's line. So scratch narrows the range of the question the define asks, from the directory to the paths in it — which is still a statement about the invocation, not about what the target does — and it says nothing L0 could act on beyond "do not ask". The cost of hiding is paid where a reader looks: the declaration is in the report and the case verbatim, the atomicity line counts what it reached, and not tested names it. The cookbook's "exempts no file by its role" was true while a define had no place to say a role.

Define has three levels; the lower ones are zero-effort:

L0 — built-in invariants, zero config. Sideeye ships general invariants it can judge without any checker. The first is atomicity, stated per file rather than per directory: for every file present in both the pre-operation snapshot and the post-operation result, the state after restart must still contain it, holding either the old content or the new one — never a mixture of the two. An agent can point sideeye at an operation and get value with no configuration at all.

The obvious wording — "the state directory must equal the pre-operation snapshot or the post-operation result" — is stricter than it looks, and wrong. A program that writes atomically leaves key.json.tmp behind in every world killed between the open and the rename, so the directory equals neither snapshot and the correct program is reported as violating the built-in invariant. Files that appear in only one of the two snapshots are therefore left unconstrained.

This is deliberately narrower than the directory-equality reading, and the difference is worth naming: L0 does not catch a crash that leaves temporary files behind forever, or one that creates a file the operation never produces on a clean run. Those are real defects, and L0 stays quiet about them. They belong to a checker (L2), or to a future built-in invariant stated in terms the correct program can actually satisfy.

L0 carries a second per-file form, chosen from the snapshots alone (ADR 0004). A file whose post-operation content strictly extends its non-empty pre-operation content — the shape of logs, journals and audit chains, whose appended bytes may legitimately differ between runs (timestamps, HMAC chains) — is judged by history preservation instead: after the crash the file must still be a file whose content begins with everything it held before the operation. The appended tail is not judged; whether a torn tail is acceptable is the target's recovery semantics, and belongs to a checker. Every report names which files were judged by which form, and not tested grows "appended tails" whenever this form is in play. A file that is empty before the operation stays on the pre-or-post rule — the preserved history of an empty file would constrain nothing.
L1 — the program's own words. Declare a success marker (a pattern on stdout). In worlds where the marker appeared before the kill, Sideeye additionally enforces the post-success invariant (§4.1).
L2 — domain checkers. A checker script for what only the author knows — for example, cross-examining a diagnostic command:

#!/bin/sh
healthy=$(mytool doctor --json | jq .healthy)
loadable=$(mytool load-key >/dev/null 2>&1 && echo true || echo false)
[ "$healthy" = "$loadable" ]   # FAIL when doctor's claim contradicts reality

The check always runs after a restart, in a fresh process. In-memory state hides corruption; v0 never evaluates invariants inside the crashed process's lifetime.

13. Output Requirements

The most important UI is the counterexample report. It ships in two forms: JSON for the caller, text for the reader. The JSON is the complete record — every field a machine might gate on, including the ones no human wants in a verdict (schema, contract_version, exit_code). The text is the reader's view of the same run, and the worked examples below are what it actually looks like: a short summary, not a second copy of the JSON. What binds them is that a value both forms carry has one definition — the JSON reads the same note the text prints, never a second formatting of it. Two things hold that: spike/check-report-schema.py's fifth claim, for every value the JSON writer passes through jsonString, and — for not_tested, which is appended directly and so falls outside that claim — a comptime table plus acceptance check 2nt, which reads one real report in both forms and requires them to agree. This sentence used to read "two forms with identical content". Its own examples below contradicted it: eleven lines for the FAIL and three for the PASS, against a document of twenty top-level fields (docs/report-schema.md carries that one; this section shows no JSON). Ruled 2026-09-01 (owner, #280) in favour of the examples.

A FAIL, as the reader sees it:

FAIL  case sideeye-000042
invariant  : check.sh exited 1  (always-invariant)
operation  : mytool rotate-key
crash point: after unlink("state/key.json"),
             before rename("state/key.json.tmp" -> "state/key.json")
             (file-op 3 of 5 — landing evidence recorded)
observed   : doctor: healthy=true / loadable key: none
expected   : doctor's claim matches reality
reproduce  : sideeye replay 000042        (reproduced 10/10)
explored   : 5 crash points, 5 restarts, 5 checks
not tested : power loss, torn writes, concurrent processes

And a PASS:

PASS  rotate-key: 5/5 explored worlds satisfied the invariant
      checker falsified before run: corrupted state -> check exited 1  [ok]
      not tested : power loss, torn writes, concurrent processes

Every report includes:

which invariant was violated,
what was expected and what was observed,
the logical crash point, with recorded evidence that the crash landed there,
the minimal reproduction steps,
measured reproducibility (not assumed),
what Sideeye did not verify,
a stable case ID for replay after the fix.

Crash points are addressed logically ("after unlink, before rename"), never by raw counters alone. When a saved case is replayed against changed code and its context no longer exists, Sideeye says "case no longer applies" — it never silently verifies a different point.

The exit-code contract

This is where §4.5 stops being prose:

exit	meaning
0	PASS — no violation found within the declared exploration
1	FAIL — reproducible violation found
2	UNKNOWN — could not judge (environment artifact, checker failed to run, unsupported target)
3	SETUP ERROR — configuration or environment problem: something the engine needed and could not get, as opposed to something it observed about the target. The JSON says which, as one of five classes in `setup_error_reason` (#518, ADR 0057), and a failing `--setup`'s status as integers. Most arise before exploration begins, and the phase is not the rule — a descriptor the parent could not arrange for a child (`/dev/null` for its stdin, the stdout capture in the work directory), a capture or oracle account it could not read back, a `fork` or an allocation that failed, all refuse this way in either phase. What they share is that the target is not what went wrong and no verdict about it is available; a refusal that is about the target, however unhelpfully, is UNKNOWN (2). This row said "before exploration began" until 2026-09-04, and the code did not: `fork`, allocation and stdin already contradicted it, and #469 made the class reachable, since a blocked capture path is the threat the work directory actually has

UNKNOWN is never 0. A caller that treats UNKNOWN as success has to do so deliberately, in its own code, against the contract.

The table maps verdicts to codes. Commands that reach no verdict are outside it: version, help and an accepting preflight all exit 0 for doing what was asked. preflight --twice (#199) adds the one negative that shape needs — exit 1 when the two observed runs left different state — which is the answer to an identity question, not a crash-consistency FAIL. The verdict-to-code mapping the freeze pins is untouched by it.

14. Functional Requirements for v0

Define one stateful operation as the verification target.
Define invariants evaluated after the operation or after restart.
Explore execution worlds where the process crashes at multiple points of the operation — process crash only.
Restart or re-evaluate the target from the crashed state.
Detect invariant violations.
Save violations as reproducible cases.
Shrink discovered cases by removing unnecessary conditions.
Replay past counterexamples as regression cases after a fix.
Never treat an unverified or unjudgeable state as success.
State explicitly, in every result, what Sideeye did and did not verify.
Start every crash world, and the recording run they are counted against, from the same rebuild of the initial state; snapshot and restore the state directory; no leakage between worlds.
Include exploration counts (crash points, restarts, check executions) in every report.
Falsify the checker once before exploring: a deliberately corrupted state must make the check FAIL; otherwise the run is UNKNOWN, not PASS.
Record crash landing context in every counterexample; on replay, report "case no longer applies" when the context no longer matches.
Emit machine-readable output (JSON) alongside human-readable text, honoring the exit-code contract of §13.
Operate fully non-interactively, suitable for CI and agent callers.

This document still does not choose OS primitives or interception techniques. It acknowledges one thing: the combination of reproducible logical crash points and native macOS + Linux support pushes the mechanism toward userspace interposition. The feasibility spike for that is the first task on the roadmap (see PRD.md).

15. Non-Goals for v0

Arbitrary syscall fault injection in general.
Network fault injection.
Clock manipulation.
Thread scheduling control.
Full deterministic execution.
Distributed system verification.
Remote service consistency.
Security vulnerability scanning.
Static analysis.
AI code review.
Formal verification.
A universal failure-modeling language.
GUI.
Cloud service.
Organization-wide chaos engineering platform.
LLM-based verdicts. Sideeye never uses a language model to decide PASS or FAIL. Not in v0, not later.
A badge. "Passed Sideeye" is not a certification, and we will not make it one. The product's output is counterexamples, not reassurance.

Being able to implement any of these is not a reason to build them before v0's value is proven.

16. What Sideeye Did Not Invent

Property-based testing, shrinking, fault injection, crash-consistency testing, deterministic simulation, and Jepsen-style adversarial verification are not Sideeye's inventions, and this document does not pretend otherwise.

The closest neighbor in spirit is ALICE (OSDI '14): ordinary applications, user-defined checkers, systematic exploration of crash states. CrashMonkey/ACE explored the filesystem side; LazyFS and dm-log-writes serve the power-failure model; QuickCheck and Hypothesis proved invariant-plus-shrinking as a developer experience.

What that lineage proved: systematically exploring worlds the developer did not imagine finds bugs ordinary tests cannot. What remains unproven — and what Sideeye exists to test — is whether that idea can become an invariant-first, everyday developer experience for ordinary stateful software. Sideeye's value is measured by how far it carries those ideas into normal development, not by hiding where they came from.

17. Success Criteria

The success metric for v0 is not feature count, fault-type count, or GitHub stars.

Primary criterion. Sideeye finds a crash-consistency bug that the target's own hand-written tests do not catch — discovered by its deterministic judge from an invariant declared and committed before this project observed any failure of the target in execution (reading a report of a failure while scouting is not observing one), with the question's provenance recorded and labeled: blind (ADR 0012's two-seal protocol) or assisted (a scout read the target; the scout and its sources named). (Redesigned 2026-08-15 in step with PRD criterion 1, ADR 0017; the prior form limited the venue to omamori, the calibration target, or a blind-protocol target — that family itself added 2026-08-13, before any blind campaign ran. Blind remains the stronger provenance; nothing assisted may borrow it, and the ordering requirement keeps timewarrior's "partial" scoring below exactly where it is.) For that bug, all of the following must hold:

Sideeye discovered it automatically.
The counterexample is reproducible.
The counterexample is small.
The author judges it a real bug or genuinely incorrect failure semantics.
After the fix, the same counterexample stops reproducing.
The counterexample is kept as a regression test.

Second criterion — the loop closes. Give a coding agent nothing but the counterexample report (JSON + replay command) and the repository. If the agent can produce a fix that makes the replay pass, without a human translating the finding, the report format has met its real audience. A report that needs human interpretation fails the positioning of §3, however correct it is.

Second criterion status (2026-08-13): met — twice, on the timewarrior finding, through both surfaces. Two different models (claude-opus-5 and claude-fable-5, both Claude 5 family) each produced a real fix, independently deriving the same three fix targets as the human patch with different implementations. Run 1 (CLI plumbing, opus): a context-free agent (fresh headless session, no memory, tool calls audited, no web use — the server-side web counters read zero) received the report, the case it names, the declared invariant the case points at, the pinned repository, and bug-blind replay plumbing — the input set is declared in the BUILDLOG protocol, and "what the report transitively names" is the honest subject, not "the report alone". The judge's own fresh replay passed (explored 2, all 24 crash points still addressed), a normal-world functional gate confirmed undo still works, and a full re-exploration of the fixed tree passed 25/25 (BUILDLOG 2026-08-13, "Loop closure: the protocol is fixed before the run", which quotes the run's manifest; the manifest itself, under spike/runs/, was never committed). Run 2 (the MCP surface, fable): the same sealed protocol with the replay carried by sideeye_replay_case, plus a third control proven first — the MCP channel itself giving opposite answers on the stage. Its gates: seal restore, fresh replay pass, non-degeneracy pass, audit clean (BUILDLOG 2026-08-13, "The MCP-mediated run", quoting its uncommitted manifest); the run-1 secondary observations (full re-exploration, upstream suites) were not repeated for run 2 — its evidence is the three gates, stated as such. Both runs: apparatus first proven by mutual-contrast controls (unpatched must fail, known patch must pass). spike/loop-closure-timew/. Since 2026-09-03 (#64) the secondary observations of a loop-closure run — a full re-exploration of the tree under the sealed define, and the four C++ suites run 1 counted — are produced by judge.sh secondary from the committed apparatus, and the generator refuses to read a run's tree until it has seen the unpatched tree fail and the known patch pass on the same stage; run 1's figures above predate it and stay as recorded, run 2 had none, and both stages are gone, and the generator's own controls on the #62 witness stage read: unpatched — FAIL, 25 worlds explored (24 crash points + baseline), earliest crash point 19 of 24, the four suites green (AtomicFileTest 18 passed / 6 skipped, data.t 96, Datafile.t 2, TagInfoDatabase.t 8); known patch — PASS 25/25, the same four suites green. Evidence beside the three gates, not a fourth gate.

First-condition closure protocol (2026-08-13). The remaining gap — "discovered automatically" held only for the crash-world search — will be measured by a blind campaign under ADR 0012: candidates, their priority order, the selection predicate, the reference rules and the audit tooling are sealed in one public commit before any candidate is executed; the invariants and checkers are sealed in a second commit after the permitted contract reading and before the first crash measurement, and exploration runs only at that second commit. Three honesty bounds are part of the claim itself: the ordering is auditable to a reader who trusts the public push history, not proven against an author who measured privately first; the sweep's sealed reports are unread by working rule, with a committed hash that only makes later substitution detectable; and the experimenter is a language model whose training may contain public information about any target — the seals make the recorded consultations and the commit order auditable, while the ledger itself is self-reported. The targets are high-risk blind targets (file-backed state, several spanning multiple files, chosen on purpose), not a second average-target calibration; §18's calibration stands on timewarrior alone.

Blind campaign result (2026-08-14, topydo 0.14). The campaign ran to completion. Seal B is 5a034aff; exploration ran from it in a clean tree and verify-seals a21b0933 5a034aff <run-manifest> <sealed-reports> returns ALL SEAL CHECKS PASSED (R1 audited) — the declaration's precedence over the first crash measurement is a machine-checked fact, at the strength the three honesty bounds above allow. Twelve of the thirteen declared operation forms produced a counterexample from the blind-declared conservation invariant; the one declared read-only form recorded no state-changing operations and passed. Scored against the six conditions:

"discovered automatically" — clean for the first time. The invariant and checker were committed before any crash world of this target existed, and the search produced the violating worlds from them. No human read a trace, no hypothesis preceded the checker. This is the condition the timewarrior finding could only score partial, and it is the reason the campaign existed.
reproducible / small / kept as a regression — clean, and stronger than timewarrior's. Each FAIL saved a case that replays in the pinned container with nothing else installed (exit 1, the case reproduced) — no target build, so these are CI-resident cases rather than a recipe (the gap #82 named; the timewarrior side has since been closed as hygiene by the timew-regression CI job, which needs a target build each run and re-records under the current contract).
"the author judges it a real bug" — open. Not asserted here. The sharpest observed behavior sits in the documented recovery path and is analysis performed after the seal, not automated discovery; spike/blind-hunt/analysis/findings.md separates the two halves explicitly.
"after the fix, the same counterexample stops reproducing" — not attempted.
Novelty — checked 2026-08-14, split verdict. The crash-window destruction is not novel as a phenomenon (topydo/topydo#318, the same failure surface under a disk-full write; this campaign adds the mechanism and a replayable case). The post-crash recovery misfire was not found in the tracker — novel as far as the recorded search sees, but it is the post-seal analysis finding, not the blind search's. No single finding holds "found by Sideeye" and "novel" at once; the PRD criterion-1 status carries the consequence.

These were high-risk blind targets, not a second average-target calibration; §18's calibration still stands on timewarrior alone.

Three selection cohorts followed (2026-08-21 #183, 2026-08-22 #209, 2026-08-23), and through all three the primary criterion stayed open (ruled met 2026-08-25 — the closing note of this paragraph). Twelve further targets under a freeze published before each ran, and twelve outcomes plus one: five walls that stood (KeePassXC at the probe; Jujutsu on static linkage; Bun on threads; cargo on a raw-syscall rename; unison at the probe, on determinism), one wall lifted by declared apparatus and turned into a verdict (Borg, #200), and seven verdicts — two null-with-verdicts where the tool's own documented contract held in every world (Mercurial 107/107, Borg 119/119), two reproduced checker-red FAILs already on their trackers, one FAIL whose earliest violating world is L0-only, one PASS, and one FAIL through a declared checker carrying checker_earliest (himalaya, cohort 4). (cargo re-measured 2026-09-16 with the released v1.4.0: the child layer no longer stands, the raw-syscall layer stands under --observe wrappers only, and under --observe syscalls cargo is judged — docs/target-classes.md.) The measurements are worth reading for what they say about the six conditions above rather than the count: "discovered automatically" and "reproducible" held throughout — from frozen define to reproduced verdict in minutes — and on 2026-08-23 the combination that had been missing arrived. himalaya is novel, automatically discovered and provenance-clean at the same time, the first finding to hold all three; what the criterion still lacked when that was written on 2026-08-23 was the author's judgement, a fix, and a regression case that runs — and upstream supplied the first two the same day, fixing the bug in io-maildir 0.3.1 and closing the report. One is left: a regression case that runs. The saved case refuses against a build carrying the fix, because the fix adds an operation the recording does not contain (spike/cohort4/himalaya-r2/upstream-fix/). Records: spike/cohort2/RESULTS.md, spike/cohort3/RESULTS.md, spike/cohort4/himalaya-r2/RESULTS.md. (Ruled 2026-08-25, owner adjudication of #305: "kept as a replayed regression case" accepts a second shape for a fix that changes the recorded operation sequence — the exhibit replays against the pinned buggy build, and the defect's absence on the fixed build is measured under controls rather than inferred. The primary criterion is met. PRD.md carries the ruling and what it deliberately gives up.)

If the primary criterion holds, Sideeye has partially mechanized a human's way of doubting — not merely re-run existing tests.

Status (2026-08-12). The primary criterion is not met on omamori and is substantially but not fully met on the calibration target §18 requires. This is the survivable pattern §18 describes ("zero findings on omamori is survivable"), not a claim that omamori's criterion was transferred. On timewarrior — a stateful CLI with no hand-written adversarial tests — Sideeye found that a crash between the three commit renames leaves timew undo deleting an interval committed before the crash. Scored against the six conditions honestly, four hold cleanly and two carry real gaps:

reproducible / small / judged a real bug / stops after the fix — clean. From the reproduce line and by hand with cp; one operation, a two-file window at crash point 14; judged real by this project's author (not yet confirmed by timewarrior's maintainers) and filed upstream as GothenburgBitFactory/timewarrior#778; a three-part patch reaches PASS 25/25 (measured).
"discovered automatically" — partial. Manual trace triage seeded the target and the window: a human read the plain strace, confirmed by hand (with cp file surgery) that undo destroys committed data, and then wrote the checker. What Sideeye automated was the crash-world search — finding and minimizing the two violating worlds of nineteen from the human-declared invariant (§4.1, "ask for invariants, invent the failures"). The mechanized half is the search, not the hypothesis.
"kept as a regression" — replayed in CI since #82 (re-recorded each run, not a committed case). The timew-regression job (.github/workflows/ci.yml) runs the committed measurement (spike/dogfood-timew-replay.sh, legs a–c) on every push to main and every pull request: build the pinned timewarrior, record the counterexample, replay it FAIL, rebuild with spike/timew-undo-ordering.patch, replay it PASS. The job re-records under the current trace contract each run, so a contract bump does not rot it. This is regression hygiene, deliberately not criterion-1 progress: under ADR 0017's ordering requirement the timewarrior finding stays "discovered automatically — partial" and does not enter criterion 1 whatever its regression form.

On omamori itself the criterion is not met; on exec, the operation this evaluation drove, it cannot be (the audit path is crash-safe by construction), and the guarded self-modification commands are not measurable under Sideeye's operation contract without break-glass — see §18.

18. Kill Criteria

Sideeye carries explicit failure conditions for itself.

If, after a defined dogfood period, Sideeye finds nothing beyond existing hand-written adversarial tests, the reasons are analyzed. The current direction is stopped or redesigned if any of the following holds:

It is only useful when humans define failure scenarios in bulk.
Defining invariants costs about as much as writing ordinary failure tests.
Counterexamples are too complex to use for actual fixes.
False positives or environment artifacts make it untrustworthy.
Reproducibility is too low for findings to serve as evidence.
Setup is too heavy for ordinary software.
No UX difference over existing specialized tools can be demonstrated.
UNKNOWN dominates: if a large share of runs on supported targets end UNKNOWN, Sideeye cannot function as a gate, whatever its detection power.

Reviewed (2026-08-16). Every condition above has been reviewed against the collected data for v1.0 entry criterion 3 — the review is docs/kill-criteria-review.md: none triggered. Row 7 (the UX difference) is scored there as an owner adjudication, its missing head-to-head disclosed; row 2 is scored on the recorded absence of the comparison it asserts; the rest are scored on committed measurements. The review reopens on any future measurement landing on a condition's trigger side.

Re-reviewed (2026-08-26, #240). The three selection cohorts' twelve targets have now been entered against all eight rows, in a dated re-review section of that same page. Six rows are corroborated or unchanged; two carry evidence on their trigger side — row 4, where Borg's three L0 violations sit in the client cache the apparatus itself relocated, and row 6, where Mercurial's define took four revisions to reach a verdict and Borg's wall fell only to a three-piece declared apparatus. The reopen condition above has fired, and the re-score is pending owner adjudication: no verdict was moved, and criterion 3's status in PRD.md was not changed. The re-review also re-checked the page's own "one instrument read eight ways" note and found it strengthened rather than weakened on the instrument axis — cohort 3 inherits cohort 2's probe gate and cohort 4 sources its predicates in place, so the three cohorts are not three independent apparatuses. Whether their twelve distinct targets and their own checkers add independent evidence is a separate question, and the note does not settle it.

Adjudicated (2026-08-27, #240): neither reopened row is triggered, and criterion 3 stays met. Row 4's case is recorded as a second example of an existing class — "tools with non-durable scratch files", precision limit #35 — rather than a new one, adding the property that a target-created scratch path can be relocated into the judged root by the measurement setup rather than sitting there from the start; it is not triggered because the file is a client cache outside the durable repository state Borg's transactional claim covers, and a checker carrying that claim ran in all 119 worlds and held in all of them. Row 4's ruling gives up two things: it does not establish that the relocated file's own documented recovery was exercised successfully (the checker attempts a deletion-and-rebuild but the call is unchecked and the committed drill records it failing on permissions), and it does not cover a relocated path judged by the built-in form with no checker carrying the target's claim at all — either of which reopens the row. Noted 2026-09-03 (ADR 0043): a define can now declare its scratch paths, so the class this row names no longer decides a verdict by itself; the basis of the ruling is unchanged, because a checker is still the only thing that says the durable state is intact and scratch adds no knowledge, only a cleaner verdict.

Row 6's ruling is a definition, and it is the one this section carries. Declared apparatus is the instrument's cost and is not setup for the purposes of the condition above. Apparatus means, narrowly: (a) declared in the public protocol before the define ran, and (b) constraining the environment the target runs in — clock, entropy source, kernel copy primitives and the like. Anything that touches the target's own installation, configuration or state is setup and is not excluded by this definition, whatever it is called in a protocol. Two further bounds, because (a) and (b) alone are not enough: the control must be measurement-only and behaviour-preserving — it may make an execution observable or repeatable, not supply behaviour the target would otherwise have to obtain — and provisioning what the target depends on is setup, including a service it needs, a database, credentials, or a stand-in for any of them, however it is injected. A shim that answers a kernel copy primitive in userspace is apparatus; a shim that emulates a network service the target requires is setup wearing a shim's clothes. The cost of revising a define until it reaches a verdict is likewise excluded from this condition by the same adjudication. Where it does belong is left open rather than assigned: the cohort-2 revisions changed state placement, a checker leg and a sendfile workaround with the question bytes unchanged, so they are measurement and define-packaging costs and are not evidence for the invariant-authoring comparison two rows above either. A usage note, because the records say the word more loosely. spike/cohort2/borg-r3/RUNLOG.md places its violating worlds in "the client cache the apparatus itself relocated", meaning the measurement setup taken as a whole. That relocation was the define's r2, not something declared before the define ran, so it does not satisfy (a) and is not excluded by this definition — it is a define revision and falls under the sentence above. Where this section says apparatus it means the narrow sense defined here, and a protocol cannot widen the exclusion by calling something apparatus after the fact. docs/kill-criteria-review.md carries the reasoning, the rejected readings and what each ruling gives up; PRD.md carries the criterion's status.

The current status of criterion 3 is carried by PRD.md's status line; the dated paragraphs above — the review, the re-review and the adjudication — are the events that produced it, and spike/check-criterion3-status.sh holds this section to pointing rather than asserting (#356).

Calibration. Judging Sideeye on omamori alone conflates "Sideeye is weak" with "omamori is hardened." The dogfood period must include at least one deliberately average target: a stateful CLI with no hand-written adversarial tests. Zero findings on omamori is survivable; zero findings on an average target is evidence for a kill.

Calibration result (2026-08-12). Both branches accounted for. On omamori, zero findings on the exec audit path — and the reconnaissance found why: the high-water mark is confirmed after the body it confirms, so verify stays conservative in every crash window (the "too hardened" reason, the survivable side). The guarded self-modification commands (config-modify, init --force, key rotate — omamori issue #12) refuse to run whether an agent or a human invokes them, so making one a Sideeye operation would require break-glass — disabling the very defence under test — and is out of scope on discipline, not measured either way; the remaining surface was enumerated exhaustively later the same day (BUILDLOG: every subcommand including nested arms, the argv0 shim mode and the hook entrypoints, cross-checked from the write-primitive side) — and re-measured under contract v10 (2026-08-16, omamori 1.0.4, #141): all four unguarded writers explore fully and hold. The v8-era refusals of install/setup/init at symlinkat/fchmodat disappeared exactly as predicted when #122 made symlinks first-class and #121 made the chmod family recorded-only, and audit verify's high-water-mark bootstrap write still passes; spike/dogfood-omamori-surface.sh pins the current outcomes and fails loudly if any moves again. On the average target — timewarrior, no hand-written adversarial tests — a real crash-consistency bug (§17 above, four of six conditions clean, two with gaps). So: hardened on the path we can drive, and a genuine find on the deliberately-average target. That is the pattern §18 says distinguishes "the tool is weak" from "the target is strong", landing on the strong-target side — which clears the calibration kill condition, not the full v1.0 entry criterion.

"We built an interesting piece of technology" is not a reason to continue.

19. Product Quality Bar

Sideeye must be as careful about its own claims as it demands others be about theirs. Phrases like "verifies crash safety" or "explores all crash points" are used only when their meaning can be strictly guaranteed.

Priority order:

Findings are real.
Findings reproduce.
Findings are small.
What was verified can be stated.
It is easy to use.
It handles many failure types.

Quality of evidence beats breadth of coverage.

20. Relationship with Ponytail

Ponytail writes it. Sideeye doesn't believe it.

Ponytail is a sibling project by the same author; the pairing is a brand contrast, not a dependency. As "making" accelerates, "doubting" has to accelerate with it. Sideeye must stand alone.

21. Long-term Direction

Only after process crash × persistent state proves real value does Sideeye consider its next malice. Candidates, in no committed order:

Power failure and torn writes — losing unsynced data, reordered writes; the natural deepening of the crash model. Considered after 1.0 and declined for now: ADR 0110 gives the reasons and what would reopen it.
Concurrent processes.
Network ambiguity.
Clock anomalies.
Partial remote execution.

The long-term vision is not a fault-type count. It is a habit: a developer writes "this property must not break," and Sideeye does the imagining, the breaking, and the shrinking.

"It went through Sideeye" will never mean "it is safe." It means exactly one thing:

This software was, at least once, seriously doubted under hostile conditions its author may not have imagined.

22. One-Sentence Definition

Sideeye is the deterministic skeptic in the coding loop: it finds the places where your software assumes a well-behaved world, breaks that assumption for real, and brings back the earliest failing crash point, saved as a replayable case.

23. The v0 Product Question

From development start until v0, every uncertain decision returns to one question:

Does this feature shorten the distance from a declared invariant to a crash bug — one the author, human or AI, would not have imagined — being found, fixed, and proven fixed, inside the loop?

Yes → it is Sideeye's core. No → at least for v0, it is not built.

FilesExpand file tree

DESIGN.md

Latest commit

History