Skip to content

RFC: build.sysroot directive for hermetic-build / manylinux targeting#343

Draft
tannevaled wants to merge 1 commit into
pkgxdev:mainfrom
tannevaled:feat/sysroot-directive
Draft

RFC: build.sysroot directive for hermetic-build / manylinux targeting#343
tannevaled wants to merge 1 commit into
pkgxdev:mainfrom
tannevaled:feat/sysroot-directive

Conversation

@tannevaled
Copy link
Copy Markdown

Status: draft / RFC

Proposes a small additive directive to the pantry yaml schema that unblocks the rest of the pantry to follow the same hermetic-build path we demonstrated empirically in pkgxdev/pantry#12968 (gnu.org/glibc).

The brewkit side is concrete (this PR, 75 lines). The pantry yaml schema docs need a companion update; opening as draft so the schema can be discussed first.

Problem

Today the pantry has no way to express "build me against THIS libc bottle, not the build host's libc". Every recipe inherits the runner's libc — so bottles carry the runner's ABI assumptions, can't target older manylinux baselines, can't be built on Alpine/musl hosts.

We worked around this empirically in pkgxdev/pantry#12968 by manually constructing the CC/CXX/CPP wrappers inside Docker containers (see pkgm/notes/session-2026-05-{19,20}.md). The pattern works — 18 glibc bottles (9 versions × 2 arches) all build host-independent — but it's recipe-side boilerplate that should be a schema feature.

Proposal

Add a build.sysroot: block:

build:
  sysroot:
    libc: gnu.org/glibc=~2.28                      # required
    kernel-headers: kernel.org/linux-headers=^7    # optional
  dependencies:
    gnu.org/glibc: ~2.28              # MUST also be a build dep
    kernel.org/linux-headers: ^7      # MUST also be a build dep

When build.sysroot.libc is set, the build script gains these exports BEFORE the user script runs:

export SYSROOT=<libc-install-prefix>
export CC="${CC:-gcc} -nostdinc -isystem <libc>/include [-isystem <khdr>/include] -B <libc>/lib -Wl,--enable-new-dtags,--dynamic-linker=<libc>/lib/ld-linux-*.so.*,--rpath=<libc>/lib"
export CXX="${CXX:-g++} <same> -nostdinc++"
export CPP="${CPP:-gcc} <same> -E"

Recipes that opt in get the routing automatically; recipes that don't are unchanged.

Behaviour change for existing recipes

None. The new helper returns an empty string when the yaml has no build.sysroot block. No existing pantry yml ships such a block.

Companion changes

Concrete use cases unlocked

  1. manylinux targeting: build.sysroot.libc: gnu.org/glibc=~2.17 produces a binary that runs on CentOS 7 / RHEL 7.
  2. Alpine / musl host build: with a musl bottle one day, build.sysroot.libc: musl.cc=*.
  3. HPC bootstrap cascade (feat(binutils): support older versions (2.27–2.38) for HPC bootstrap cascade pantry#12966, #12967, #12968): the gnu.org/glibc recipe for older versions becomes a single yml block instead of the manual Dockerfile dance.
  4. dist.pkgx.dev parity with Nix nixpkgs: every package gets a known glibc baseline.

Open questions

  1. Schema bikeshed: build.sysroot.libc vs build.libc? build.sysroot.libc vs separate top-level sysroot:?
  2. Should the directive imply the dep (auto-add to build.dependencies)? Current sketch requires the recipe to declare both for clarity.
  3. Should the wrapper include -D_FORTIFY_SOURCE=… or similar hardening flags by default? Currently passthrough.
  4. How does this interact with darwin? Initial implementation: host().arch switch covers x86-64 / aarch64 linux; darwin throws. Should it be a no-op on darwin instead?

Happy to iterate on the schema, split into smaller PRs, or rework as needed.

Refs

🤖 Generated with Claude Code

Adds a `build.sysroot:` block to the package.yml schema that, when
set, redirects the compiler at a specific glibc / kernel-headers
bottle (instead of using the build host's libc). This is the
hermetic-build / host-independent piece needed for the pantry to
target manylinux baselines, build on Alpine-musl hosts, and produce
bottles that don't carry the build runner's libc ABI assumptions.

YAML schema:

    build:
      sysroot:
        libc: gnu.org/glibc=~2.28               # required
        kernel-headers: kernel.org/linux-headers=^7   # optional
      dependencies:
        gnu.org/glibc: ~2.28      # MUST also be a build dep
        kernel.org/linux-headers: ^7  # MUST also be a build dep

When `build.sysroot.libc` is set, the generated build script exports:

    SYSROOT=<libc-install-prefix>
    CC="${CC:-gcc} -nostdinc -isystem <libc>/include [-isystem <khdr>/include] -B <libc>/lib -Wl,--enable-new-dtags,--dynamic-linker=<libc>/lib/ld-linux-*.so.*,--rpath=<libc>/lib"
    CXX="${CXX:-g++} <same> -nostdinc++"
    CPP="${CPP:-gcc} <same> -E"

The libc package must already be a `build.dependencies` entry (so
it's installed and resolved); this directive merely routes existing
deps, it doesn't add them. Same constraint for `kernel-headers`.

No behaviour change for recipes that don't declare `build.sysroot:` —
the helper returns an empty string and no env exports are emitted.

This is the brewkit half of the feature; the pantry yaml schema docs
need a companion update (will land in a pkgxdev/pantry PR).

Empirical motivation: pkgxdev/pantry#12968 (gnu.org/glibc) and the
HPC-cascade work in pkgm/notes/session-2026-05-{19,20}.md. We
demonstrated that this exact sysroot routing pattern (done manually
inside Docker containers) enables building glibc 2.17–2.43 host-
independent on both linux/x86-64 and linux/aarch64 from a clean
debian:bookworm-slim with no apt-installed compiler. Formalising it
in the recipe schema unblocks the rest of the pantry to follow the
same hermetic-build path.
@jhheider
Copy link
Copy Markdown
Contributor

should this be linux-sysroot? or should we just base it on the presence of a glibc dep? as for CC config, look into the scripts/symlinks already produced under libexec. we can either make them smarter, or have alternates for sysrooted variants.

@tannevaled
Copy link
Copy Markdown
Author

On naming: agreed linux-sysroot is better than build.sysroot.libc:

  • A sysroot is more than just libc (headers, crt, optional gcc bits, …)
  • Bundling it under linux-* keeps the platform context explicit at read-time

On implicit-from-glibc-dep: I'd argue against, two reasons:

  • Some recipes (Go binaries with cgo disabled, kernel modules) want NO glibc at runtime even if glibc creeps into the build closure via host tools
  • An explicit linux-sysroot: gnu.org/glibc@2.17 makes intent readable instead of inferring from the dep graph

So: explicit directive, sibling to other build-time knobs, named linux-sysroot.

On extending libexec/ scripts: yes — looked at the existing CC/CXX wrappers there. The right factoring seems to be:

  • Build-time (this PR's territory): linux-sysroot: gnu.org/glibc@X.Y → brewkit's existing CC wrapper detects the env-injected LINUX_SYSROOT and adds -nostdinc -isystem $SYSROOT/include -B $SYSROOT/lib -L $SYSROOT/lib -Wl,--rpath=$SYSROOT/lib automatically. No new wrapper needed — just smarten the existing one.
  • Install-time (separate, opt-in): bklibcvenv seal {{prefix}} (bklibcvenv: hermetic glibc helper (analogue to bkpyvenv) #344) wraps bin/* into libexec/* for runtime hermeticity. Independent of the build-time sysroot piece.

The composition: build-time sysroot makes you link against a specific libc; install-time bklibcvenv makes you run against it. Both compose, but neither requires the other.

Will refactor this PR around the linux-sysroot name and the CC-wrapper extension if you confirm that direction. (Specifically: which file in libexec/ already handles CC flag injection — libexec/cc doesn't exist as such, so I'd guess the logic lives in the brewkit env-setup TypeScript code, not a shell wrapper. Pointer welcome.)

@jhheider
Copy link
Copy Markdown
Contributor

The point about the default is that we always have glibc now, so if we instead build against glibc2.17 by default, we get greater compatibility, and we only need to override it if we don't care (go, maybe, but it does no harm), or to change the value by adding a dep. Then we didn't need the sysroot idea at all, we're just generally building against 2.17 for max compatibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants