bootstrap: add RUSTC_EXTRA_REMAP for reproducible host builds

[DeepeshWR]

Right now, rust.remap-debuginfo = true doesn't completely remap all paths embedded in compiled artifacts. While source directories are remapped via RUSTC_DEBUGINFO_MAP (to /rustc/$sha for compiler sources, virtual paths for std) and Cargo registry sources are remapped via RUSTC_CARGO_REGISTRY_SRC_TO_REMAP (to /rust/deps), the build output directory is not remapped for host compilations.

The existing remapping coverage:

RUSTC_DEBUGINFO_MAP:
- compiler/ -> /rustc-dev/$sha/compiler
- $src_dir -> /rustc-dev/$sha
- library/ -> /rustc/$sha/library

RUSTC_CARGO_REGISTRY_SRC_TO_REMAP:
- $CARGO_HOME/registry/src/* -> /rust/deps

RUSTFLAGS (via Cargo):
- Covers --remap-path-prefix for target compilations only

What is NOT covered:

Build output directories (where OUT_DIR lives) for host compilations.
Cargo only passes RUSTFLAGS to target builds (rust-lang/cargo#4423).
The bootstrap rustc shim only passes RUSTC_HOST_FLAGS (linker args)
and RUSTC_DEBUGINFO_MAP (source paths) to host builds. There is no
mechanism to remap the build output directory for host compilations.

This matters because crates like thiserror and rustc_macros use build scripts that generate source files in OUT_DIR via include!(). The absolute OUT_DIR path (e.g.,
build/x86_64-.../stage0-rustc//out/) gets embedded into the crate's SVH (Strict Version Hash). With -Zdual-proc-macros enabled during cross-compilation, host proc macros carry these unremapped paths, causing the SVH to differ between builds with different build directories. This cascades into rustc_driver and other final artifacts, breaking reproducibility.

Fix this by adding support for a RUSTC_EXTRA_REMAP environment variable in the rustc shim. Like RUSTC_DEBUGINFO_MAP and
RUSTC_CARGO_REGISTRY_SRC_TO_REMAP, it uses tab-separated from=to mappings and applies --remap-path-prefix to all compilations (both host and target). This allows external build systems to remap paths that bootstrap doesn't know about (such as out-of-tree build directories used in cross-compilation environments like Yocto/OpenEmbedded and Nix).

The variable is only active when explicitly set, so it has no effect on default builds or CI.

[rustbot]

r? @clubby789

rustbot has assigned @clubby789.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: bootstrap
• bootstrap expanded to 6 candidates
• Random selection from Mark-Simulacrum, clubby789, jieyouxu

[clubby789]

Does using RUSTFLAGS_BOOTSTRAP not solve the issue for you?

[DeepeshWR]

Does using RUSTFLAGS_BOOTSTRAP not solve the issue for you?

No, RUSTFLAGS_BOOTSTRAP does not solve this issue.

RUSTFLAGS_BOOTSTRAP gets merged into RUSTFLAGS (via propagate_rustflag_envs in src/bootstrap/src/core/builder/cargo.rs:67), which is then set as the RUSTFLAGS env var on the Cargo invocation (line 458).

However, bootstrap always passes --target to Cargo (line 554):

  if cmd_kind != Kind::Install {
      cargo.arg("--target").arg(target.rustc_target_arg());
  }

Per Cargo's documented behavior (doc):

If the --target flag (or build.target) is used, then the flags will only be passed to the compiler for the target. Things being built for the host, such as build scripts or proc macros, will not receive the args.

So any --remap-path-prefix flags added via RUSTFLAGS_BOOTSTRAP will never reach host proc-macro compilations (thiserror,rustc_macros, etc.) which are exactly the crates embedding unremapped OUT_DIR paths into their SVH.

RUSTC_EXTRA_REMAP solves this by being read in the rustc shim (src/bootstrap/src/bin/rustc.rs) after the host/target branch the same location where RUSTC_DEBUGINFO_MAP and RUSTC_CARGO_REGISTRY_SRC_TO_REMAP are already applied unconditionally to all compilations. It follows the existing pattern.

[bjorn3]

Instead of adding an env var that the user needs to set, I think bootstrap itself should determine the necessary remapping. It knows where the build dir is.

[DeepeshWR]

@bjorn3 Thanks for the pointers! I had an idea to extend the remap-debuginfo functionality to also cover self.build.out, which is used for OUT_DIR, but it looks like that fix has already landed in #155935 and is available starting with Rust 1.97.0. Upgrading from 1.96.0 to 1.97.0 resolved the issue.