[DerDennisOP]

Motivation

I maintain a small CI system that reads the CPU, memory and disk usage of nix builds. With #16024 the build cgroups expose memory.peak, io.stat and memory.events, but BuildResult only has the CPU times and even those were lost on the way to daemon clients.

Context

Five commits.

• Stats for completed builds live in DerivationTrampolineGoal, so timesBuilt, build time and CPU usage reach Worker::buildDerivation() clients.
• Peak memory, peak swap, block I/O bytes and the number of OOM kills form one map from stat name to number in getCgroupStats().
• Length-prefixed name/value pairs carry that map over the worker protocol when both sides negotiate build-resource-usage. New stats need no protocol change. Golden tests pin the bytes.
• Failed builds record the number of processes the kernel killed due to OOM.
• Stats appear under resourceUsage in nix build --json.

VM tests for tests/nixos/cgroups confirm that memoryPeak reach the clients. They depend on #16744.

---

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[xokdvium]

But JSON build result isn't actually user visible. I think it's only there for tests for whatever reason? So none of these changes are actually user-visible.

[xokdvium]

I guess it might make a ton of sense to include these results in nix build --json output

[DerDennisOP]

nah that's still from my patch notes. build-resource-usage feature is a thing Gradient does. But Yeah let's put it in nix build --json output.

[xokdvium]

How are we going to keep this stuff extensible? Could we just have a list of keys and some length-prefixed format of key/value pairs maybe?

[Lillecarl]

Thanks for this. I had been working on the same statistics in my fork, so here is what I ran into. Take what helps.

Daemon builds get none of these values on master today. Since the self-exec daemon (#16659), the worker starts inside the nix-daemon sub-cgroup, so getRootCgroup() returns nix-daemon and build cgroups are created inside it. nix-daemon holds processes, so the controllers from #16024 cannot reach the builds, and memory.peak, io.stat and memory.events do not exist there. tests/nixos/cgroups on master shows it. #16744 fixes that; with it, this PR should report real numbers for daemon builds.

memory.swap.peak. cgroup v2 takes pages that go to swap out of the memory count and adds them to the swap count. On a host that swaps, memoryPeak alone is lower than what the build needed. Reporting memorySwapPeak next to it (Linux 6.5+, absent without swap support) gives a scheduler the full figure.

nix build --json. src/nix/build.cc prints cpuUser and cpuSystem from the BuildResult but not the new fields, so nix build users cannot see them yet.

Protocol test. buildResult_resource_usage_round_trip writes and reads back with the same code, so a bug that is the same on both sides (a swapped field order, say) still passes. A VERSIONED_CHARACTERIZATION_TEST with golden data, like the other BuildResult cases in that file, pins the bytes on the wire.

Read errors. readCgroupFile checks pathExists and then calls readFile, which throws if the read fails. Should a failure to read a statistics file fail the build? Catching SystemError there and leaving the field empty seems safer.

Rebase. src/libutil-tests/linux/meson.build and the subdir('linux') line already exist on master, so GitHub reports a conflict.

I have code for the swap peak, the nix build --json output, a golden protocol test and a VM assertion on memoryPeak (including that an already-valid path reports none). I can send any of them as suggestions here or as a follow-up PR on top of this one, whichever you prefer.

---

Drafted with Claude Code (claude-opus-5-5) and reviewed by me; the points above were checked against the diff and current master.

[Lillecarl]

Following up with links, in case any of it is useful. My branch is Lillecarl#284 (description has the reasoning and the mutation tests). The relevant commits:

• memory.peak and memory.swap.peak parsing, with tests for zero versus absent, unparsable values and a missing swap file: Lillecarl@d1b9d55
• the protocol feature with a golden characterization test: Lillecarl@95c2349
• nix build --json output, release note and the tests/nixos/cgroups assertions: Lillecarl@cdc52f3

Golden test for build-resource-usage. Add a case next to the other BuildResult cases in src/libstore-tests/worker-protocol.cc:

VERSIONED_CHARACTERIZATION_TEST(
    WorkerProtoTest,
    buildResult_resourceUsage,
    "build-result-resource-usage",
    (WorkerProto::Version{
        .number = {.major = 1, .minor = 38},
        .features = {"build-resource-usage"},
    }),
    ({
        std::tuple<BuildResult, BuildResult> t{
            /* one result with every new field set, one with none */
        };
        t;
    }))

Then let the test write its expected data and check it in:

$ _NIX_TEST_ACCEPT=1 meson test nix-store-tests -v   # writes the golden files; these tests report SKIPPED
$ meson test nix-store-tests -v                      # now they compare against them and pass
$ git add src/libstore-tests/data/worker-protocol/build-result-resource-usage.*

The macro creates _read and _write tests (plus JSON ones), so a field written in the wrong order or read back wrongly fails against the checked-in bytes even if both sides agree with each other. Mine has a version of this at buildResult_memory; the features list should include whatever else defaultVersion has on your base (mine also needed realisation-with-path-not-hash).

---

Drafted with Claude Code (claude-opus-5-5) and reviewed by me.

[DerDennisOP]

Thanks for this. I had been working on the same statistics in my fork, so here is what I ran into. Take what helps.
[...]
Drafted with Claude Code (claude-opus-5-5) and reviewed by me; the points above were checked against the diff and current master.

@Lillecarl Could you please tell us in 2 sentences what you want to tell us maybe?

[Lillecarl]

@DerDennisOP

• record swapping
• print new results to nix build --json
• record new protocol test

I authored the previous comments to be helpful for an AI agent, I'm a creepy stalker and noticed you work for an AI company 😄

A review on #16744 (just fresh) that fixes cgroups after a recent regression would be nice if you can spare the time