Merge origin/main into agent/fix-vendor-check-wiring-liveness

Brings in the vex_consumed alias test fix (#849) that main's red
test/test-release/coverage jobs were waiting on.

Co-Authored-By: Claude <noreply@anthropic.com>

Author: mikolalysenko

AGENTS.md

@@ -0,0 +1,13 @@
+# Agent instructions
+
+## CHANGELOG.md is written only at release time
+
+Don't add, edit or delete entries in `CHANGELOG.md` in a feature, fix,
+refactor, CI or docs PR, even if a doc, template or reviewer asks for one.
+The `[Unreleased]` section is written when a release is cut, by the release
+agent, from the PRs merged since the last tag and the code itself. Put the
+details of a change in its commit messages and PR description instead.
+
+If a PR you work on already changes `CHANGELOG.md`, restore that file to its
+merge-base version. The only exceptions are release PRs: a `release/v*`
+branch, or the release train's `release-sync` PR.

CHANGELOG.md

@@ -9,9 +9,11 @@ Pre-v3.0 entries are concise summaries derived from each tag's commit
 history. For full per-release detail, see the
 [GitHub releases page](https://github.com/SocketDev/socket-patch/releases).
 
-Add entries under `[Unreleased]`; its `###` headings set the next version's
-bump (Breaking/Removed → major, Added/Changed/Deprecated → minor, anything
-else → patch). Releases are cut by the release train
+PRs never edit this file. Only the release agent writes `[Unreleased]`, at
+release time, from the PRs merged since the last tag and the code they
+changed. Its `###` headings set the next version's bump (Breaking/Removed →
+major, Added/Changed/Deprecated → minor, anything else → patch). Releases
+are cut by the release train
 ([docs/release-train/DESIGN.md](docs/release-train/DESIGN.md)) with
 `scripts/release.py`: a release candidate's `[Unreleased]` entries become a
 `## [X.Y.Z-rc.N]` section, the rolling `release-sync` PR brings each cut

CLAUDE.md

@@ -0,0 +1 @@
+@AGENTS.md

crates/socket-patch-cli/CLI_CONTRACT.md

@@ -387,7 +387,7 @@ Recognition rules that hold for every ecosystem:
 |---|---|---|
 | Vendored: a lockfile/config wires a `.socket/vendor` artifact, or a live vendor ledger entry | The **committed artifact** is hashed against the record's `afterHash`. The ledger entry is used when it names the wired artifact (it carries the dir-artifact inventory); otherwise an entry is synthesized from the reference. A present installed tree with different bytes only warns `vendored_tree_out_of_sync`. | `(vendored)` |
 | Hosted: a discovered patch-host reference (or a live pre-v5 redirect-ledger record) | The installed copies the build **consumes** through the hosted wiring are hash-verified when any exist: the Go replacement module, never the pristine `M@v` in the module cache; the Socket-registry cargo source dir; maven's suffixed version. Installed evidence wins: `hash_mismatch` / `not_applied` are omitted. With **nothing installed**, a discovered reference whose lock pins the artifact (or whose format's rewriter never writes a pin) attests from that pin, which is the same evidence as in-run `scan --mode hosted --vex`. A pre-v5 ledger-only record, or a reference whose required pin is missing, stays `package_not_found`. So do purls that `--ecosystems` kept out of the crawl, because "not installed" has to mean the crawler looked. The same goes for npm purls when an installed pnpm tree records its virtual store outside the project (`enableGlobalVirtualStore`, or a `virtualStoreDir` that climbs out): transitive deps there are invisible to the crawler. A pnpm `modulesDir` inside the project is crawled. | `(redirected)` |
-| Agent: a manifest record with no live hosted/vendored wiring | The installed tree, unchanged. **Every** installed copy the crawler finds for the purl (npm nests duplicates of one `name@version`) must hash to the patched bytes, as `apply` patches every copy. One unpatched copy omits the purl with that copy's tag (`not_applied` / `hash_mismatch`). | none |
+| Agent: a manifest record with no live hosted/vendored wiring | The installed tree, unchanged. **Every** installed copy the crawler finds for the purl (npm nests duplicates of one `name@version`; pnpm, vlt, Bun and Deno stores add peer-variant copies and copies bundled inside other packages) must hash to the patched bytes, as `apply` patches every copy. One unpatched copy omits the purl with that copy's tag (`not_applied` / `hash_mismatch`). | none |
 
 **Liveness gates.** These gates run before hashing, and `--no-verify` / `--vex-no-verify` skips only the hashing, never the gates:
 

crates/socket-patch-cli/src/commands/vex_consumed.rs

@@ -40,7 +40,8 @@
 use std::collections::{BTreeMap, HashMap};
 use std::path::{Path, PathBuf};
 
-use socket_patch_core::crawlers::npm_crawler::find_store_peer_variant_copies;
+#[cfg(not(test))]
+use socket_patch_core::crawlers::npm_crawler::with_store_peer_variant_copies;
 use socket_patch_core::crawlers::{
     CargoCrawler, CrawlerOptions, Ecosystem, GoCrawler, MavenCrawler, NpmCrawler,
 };
@@ -50,6 +51,8 @@ use socket_patch_core::vendor::go_mod_edit::{
 };
 use socket_patch_core::vendor::lock_inventory::LockIntegrity;
 use socket_patch_core::vex::HostedCopies;
+#[cfg(test)]
+use tests::recording_store_variants as with_store_peer_variant_copies;
 
 use crate::args::GlobalArgs;
 use crate::commands::vex_sources::HostedWiring;
@@ -61,8 +64,9 @@ use crate::ecosystem_dispatch::{
 /// module docs), under the same crawler options and `--ecosystems` scope as
 /// the installed-tree lookup. `installed` is that lookup's every-copy
 /// result ([`crate::ecosystem_dispatch::find_manifest_package_copies_reusing`] over
-/// the record view, which holds every hosted purl): the shared-location
-/// ecosystems read it instead of crawling the tree a second time. `prior`
+/// the record view, which holds every hosted purl), including npm store
+/// variants. The shared-location ecosystems read it instead of crawling
+/// the tree a second time. `prior`
 /// (embedded hosted `scan --vex` only) is scan's npm crawl of the same
 /// tree: the alias walk takes its `node_modules` roots and the identity
 /// fallback its packages instead of walking the tree again.
@@ -109,13 +113,38 @@ pub(crate) async fn hosted_consumed_copies(
             let mut paths = all.remove(purl).unwrap_or_default();
             // The installed-tree lookup already resolves importer-tree
             // aliases, so most of the walk's finds are in `paths` already.
-            for alias in aliases.remove(purl).unwrap_or_default() {
-                if !paths.contains(&alias) {
-                    paths.push(alias);
+            let extra: Vec<PathBuf> = aliases
+                .remove(purl)
+                .unwrap_or_default()
+                .into_iter()
+                .filter(|alias| !paths.contains(alias))
+                .collect();
+            if npm.contains(&purl) && installed.get(purl).is_some_and(|p| !p.is_empty()) {
+                // The installed lookup already expanded these copies.
+                // Expanding its N variants again scans the store N times.
+                // Only aliases are new; expand them before merging so a
+                // different alias/store can still contribute more copies.
+                if !extra.is_empty() {
+                    let added = with_store_peer_variant_copies(extra).await;
+                    let mut seen = std::collections::HashSet::new();
+                    for path in &paths {
+                        seen.insert(tokio::fs::canonicalize(path).await.unwrap_or(path.clone()));
+                    }
+                    for path in added {
+                        let canonical =
+                            tokio::fs::canonicalize(&path).await.unwrap_or(path.clone());
+                        if seen.insert(canonical) {
+                            paths.push(path);
+                        }
+                    }
                 }
-            }
-            if npm.contains(&purl) {
-                paths = with_store_variants(paths).await;
+            } else if npm.contains(&purl) {
+                // No installed copies: the identity fallback and aliases
+                // have not had their store variants enumerated yet.
+                paths.extend(extra);
+                paths = with_store_peer_variant_copies(paths).await;
+            } else {
+                paths.extend(extra);
             }
             out.insert(
                 purl.clone(),
@@ -321,28 +350,6 @@ async fn npm_identity_fallback_reusing(
     }
 }
 
-/// `paths` plus every store variant of each (a pnpm peer suffix, a vlt peer
-/// or modifier extra, a vlt registry-alias instance of the same
-/// `name@version`): the crawler resolves a store copy only for a package
-/// with no importer copy, leaving the variants to apply's fan-out, but each
-/// variant is what some dependent loads.
-async fn with_store_variants(paths: Vec<PathBuf>) -> Vec<PathBuf> {
-    let mut seen: std::collections::HashSet<PathBuf> = std::collections::HashSet::new();
-    for path in &paths {
-        seen.insert(tokio::fs::canonicalize(path).await.unwrap_or(path.clone()));
-    }
-    let mut out = paths.clone();
-    for path in &paths {
-        for copy in find_store_peer_variant_copies(path).await {
-            let canonical = tokio::fs::canonicalize(&copy).await.unwrap_or(copy.clone());
-            if seen.insert(canonical) {
-                out.push(copy);
-            }
-        }
-    }
-    out
-}
-
 // ── golang ───────────────────────────────────────────────────────────────
 
 /// Under `replace M v => patch.socket.dev/gopatch/<uuid> <sver>` the build
@@ -604,6 +611,219 @@ async fn maven_copies(options: &CrawlerOptions, purl: &str, wiring: &HostedWirin
 mod tests {
     use super::*;
 
+    tokio::task_local! {
+        // Observe real expansion work only in the regression's own task;
+        // concurrent tests keep calling the production helper normally.
+        static VARIANT_INPUTS: std::cell::RefCell<Vec<Vec<PathBuf>>>;
+    }
+
+    pub(super) async fn recording_store_variants(paths: Vec<PathBuf>) -> Vec<PathBuf> {
+        let _ = VARIANT_INPUTS.try_with(|calls| calls.borrow_mut().push(paths.clone()));
+        socket_patch_core::crawlers::npm_crawler::with_store_peer_variant_copies(paths).await
+    }
+
+    #[cfg(unix)]
+    async fn tracked_npm_hosted(
+        common: &GlobalArgs,
+        installed: &HashMap<String, Vec<PathBuf>>,
+    ) -> (Vec<PathBuf>, Vec<Vec<PathBuf>>) {
+        let purl = "pkg:npm/left-pad@1.3.0".to_string();
+        let hosted = BTreeMap::from([(
+            purl.clone(),
+            HostedWiring {
+                uuid: "11111111-1111-4111-8111-111111111111".to_string(),
+                refs: Vec::new(),
+            },
+        )]);
+        VARIANT_INPUTS
+            .scope(std::cell::RefCell::new(Vec::new()), async {
+                let mut found = hosted_consumed_copies(common, &hosted, installed, None).await;
+                let paths = found.remove(&purl).unwrap().paths;
+                let calls = VARIANT_INPUTS.with(|inputs| inputs.borrow().clone());
+                (paths, calls)
+            })
+            .await
+    }
+
+    #[cfg(unix)]
+    fn peer_copies(store: &Path, count: usize) -> Vec<PathBuf> {
+        (0..count)
+            .map(|i| {
+                let path = store.join(format!(
+                    "left-pad@1.3.0(peer@1.0.{i})/node_modules/left-pad"
+                ));
+                pkg(&path, "left-pad", "1.3.0");
+                path
+            })
+            .collect()
+    }
+
+    #[cfg(unix)]
+    #[tokio::test]
+    async fn hosted_reuses_expanded_npm_copies_and_merges_alias_variants() {
+        let tmp = tempfile::tempdir().unwrap();
+        let nm = tmp.path().canonicalize().unwrap().join("node_modules");
+        let peers = peer_copies(&nm.join(".pnpm"), 8);
+        std::os::unix::fs::symlink(&peers[0], nm.join("left-pad")).unwrap();
+        let common = GlobalArgs {
+            cwd: tmp.path().canonicalize().unwrap(),
+            ecosystems: Some(vec!["npm".to_string()]),
+            ..GlobalArgs::default()
+        };
+        let purl = "pkg:npm/left-pad@1.3.0".to_string();
+        let installed = crate::ecosystem_dispatch::find_manifest_package_copies_reusing(
+            std::slice::from_ref(&purl),
+            &common,
+            true,
+            None,
+        )
+        .await;
+        assert_eq!(installed[&purl].len(), peers.len());
+        let (paths, calls) = tracked_npm_hosted(&common, &installed).await;
+        assert_eq!(paths, installed[&purl]);
+        assert!(
+            calls.is_empty(),
+            "already-expanded copies were rescanned: {calls:?}"
+        );
+
+        // A real alias is absent from the name-keyed installed set. Its
+        // store variants overlap that set canonically, including the
+        // importer link's physical copy; keep the alias once and preserve
+        // the original importer-first path choices.
+        let alias = nm.join("lp");
+        pkg(&alias, "left-pad", "1.3.0");
+        let (paths, calls) = tracked_npm_hosted(&common, &installed).await;
+        assert_eq!(calls, vec![vec![alias.clone()]]);
+        let mut expected = installed[&purl].clone();
+        expected.push(alias);
+        assert_eq!(paths, expected);
+
+        // An alias beneath a real nested host can reach another store.
+        // The installed root copy makes the name-keyed resolver skip
+        // those peers, so alias expansion must still add them even when
+        // installed copies are already present.
+        let host = nm.join("host");
+        pkg(&host, "host", "1.0.0");
+        let host_nm = host.join("node_modules");
+        let nested_peers = peer_copies(&host_nm.join(".pnpm"), 2);
+        let nested_alias = host_nm.join("lp");
+        pkg(&nested_alias, "left-pad", "1.3.0");
+        let installed_again = crate::ecosystem_dispatch::find_manifest_package_copies_reusing(
+            std::slice::from_ref(&purl),
+            &common,
+            true,
+            None,
+        )
+        .await;
+        // Since #605 the name-keyed resolver probes bundled trees itself, so
+        // it already returns the aliases and the nested store's peers. Feed
+        // the earlier, alias-free set to keep exercising alias expansion;
+        // the resolver's own set is checked against the same result below.
+        let (paths, calls) = tracked_npm_hosted(&common, &installed).await;
+        assert_eq!(calls.len(), 1);
+        let mut inputs = calls[0].clone();
+        inputs.sort();
+        let mut aliases = vec![nm.join("lp"), nested_alias.clone()];
+        aliases.sort();
+        assert_eq!(inputs, aliases);
+        assert_eq!(&paths[..installed[&purl].len()], installed[&purl]);
+        expected.push(nested_alias);
+        expected.extend(nested_peers);
+        let mut actual = paths.clone();
+        actual.sort();
+        expected.sort();
+        assert_eq!(actual, expected);
+        assert_eq!(
+            paths
+                .iter()
+                .map(|path| path.canonicalize().unwrap())
+                .collect::<std::collections::HashSet<_>>()
+                .len(),
+            paths.len()
+        );
+        let (mut resolved, _) = tracked_npm_hosted(&common, &installed_again).await;
+        resolved.sort();
+        assert_eq!(resolved, expected, "the resolver's own copy set");
+    }
+
+    #[cfg(unix)]
+    #[tokio::test]
+    async fn hosted_expands_alias_only_copies() {
+        let tmp = tempfile::tempdir().unwrap();
+        let store = tmp
+            .path()
+            .canonicalize()
+            .unwrap()
+            .join("node_modules/.pnpm");
+        let peers = peer_copies(&store, 2);
+        // Run within a store package whose nested dependency is an alias.
+        // The sibling peer copies are outside its project-root search.
+        let root = store.join("host@1.0.0/node_modules/host");
+        let alias = root.join("node_modules/lp");
+        pkg(&alias, "left-pad", "1.3.0");
+        let common = GlobalArgs {
+            cwd: root,
+            ecosystems: Some(vec!["npm".to_string()]),
+            ..GlobalArgs::default()
+        };
+        let purl = "pkg:npm/left-pad@1.3.0".to_string();
+        let installed = crate::ecosystem_dispatch::find_manifest_package_copies_reusing(
+            std::slice::from_ref(&purl),
+            &common,
+            true,
+            None,
+        )
+        .await;
+        // Since #605 the name-keyed resolver reaches the alias and its
+        // sibling peers on its own. An alias-only set (what an alias-blind
+        // resolver returns) must still expand to the same copies.
+        let (mut paths, calls) = tracked_npm_hosted(&common, &HashMap::new()).await;
+        assert_eq!(calls, vec![vec![alias.clone()]]);
+        let mut expected = peers;
+        expected.push(alias);
+        paths.sort();
+        expected.sort();
+        assert_eq!(paths, expected);
+        let (mut resolved, _) = tracked_npm_hosted(&common, &installed).await;
+        resolved.sort();
+        assert_eq!(resolved, expected, "the resolver's own copy set");
+    }
+
+    #[cfg(unix)]
+    #[tokio::test]
+    async fn hosted_expands_identity_fallback_with_empty_installed_entry() {
+        let tmp = tempfile::tempdir().unwrap();
+        let peers = peer_copies(
+            &tmp.path()
+                .canonicalize()
+                .unwrap()
+                .join("external/node_modules/.pnpm"),
+            2,
+        );
+        let root = tmp.path().canonicalize().unwrap().join("project");
+        let alias = root.join("node_modules/lp");
+        std::fs::create_dir_all(alias.parent().unwrap()).unwrap();
+        std::os::unix::fs::symlink(&peers[0], &alias).unwrap();
+        let common = GlobalArgs {
+            cwd: root,
+            ecosystems: Some(vec!["npm".to_string()]),
+            ..GlobalArgs::default()
+        };
+        let purl = "pkg:npm/left-pad@1.3.0".to_string();
+        assert!(
+            npm_alias_copies(&common.crawler_options(), std::slice::from_ref(&purl))
+                .await
+                .is_empty()
+        );
+        let installed = HashMap::from([(purl, Vec::new())]);
+        let (mut paths, calls) = tracked_npm_hosted(&common, &installed).await;
+        assert_eq!(calls, vec![vec![alias.clone()]]);
+        let mut expected = vec![alias, peers[1].clone()];
+        paths.sort();
+        expected.sort();
+        assert_eq!(paths, expected);
+    }
+
     fn pkg(dir: &Path, name: &str, version: &str) {
         std::fs::create_dir_all(dir).unwrap();
         std::fs::write(
@@ -857,7 +1077,7 @@ mod tests {
             nm.join("left-pad"),
         )
         .unwrap();
-        let mut got = with_store_variants(vec![nm.join("left-pad")]).await;
+        let mut got = with_store_peer_variant_copies(vec![nm.join("left-pad")]).await;
         got.sort();
         let mut want = vec![
             nm.join("left-pad"),
@@ -866,7 +1086,7 @@ mod tests {
         ];
         want.sort();
         assert_eq!(got, want);
-        assert!(with_store_variants(Vec::new()).await.is_empty());
+        assert!(with_store_peer_variant_copies(Vec::new()).await.is_empty());
     }
 
     /// vlt twin of the `.pnpm` case: every importer entry is a link into