Fix gem takeover un-hosting a grouped gem (#775)

crates/socket-patch-cli/CLI_CONTRACT.md

@@ -120,7 +120,7 @@ For a **9.0 root lock**, the CLI ensures `pnpm-workspace.yaml` carries `trustLoc
 
 **vlt hosted-mode contract**: `scan` / `get --mode hosted` rewrite, in `vlt-lock.json`, every default-registry node of a granted `name@version` (the `''` / `npm` segment or a URL segment equal to the lock's scalar `registry`, both DepID grammars, every peer and modifier variant): slot [2] becomes the granted sha512 and slot [3] the hosted URL (appended to a 3-tuple); the DepID, flags and trailing slots, the line ending and every other byte stay. `options` is never edited and `vlt.json` is only read. A lock with another `lockfileVersion` (decided on the raw JSON token), a BOM, a non-object body or a `nodes` section outside vlt's one-node-per-line layout refuses the whole lock (`redirect_vlt_lock_unsupported`). **Confirmation**: vlt drives when its install state (`node_modules/.vlt-lock.json` or `node_modules/.vlt/`) is present or no other npm-family lock is; then only `vlt-lock.json` confirms a uuid. Otherwise every lock is rewritten, `redirect_vlt_sibling_lockfiles` warns, and the other locks' rules confirm, including a dep `vlt-lock.json` merely does not wire (`redirect_vlt_entry_not_found`, `redirect_vlt_entry_vendored`). Whichever lock drives, a dep the vlt rewriter refuses (`redirect_vlt_missing_sha512`, `redirect_vlt_unsupported_lock_key`) is never confirmed by any lock, although a sibling lock may already carry its rewritten URL. **Artifact preflight**: before any takeover or write (dry runs included), each granted artifact with a default-registry instance is fetched once as vlt fetches it and must verify, else the dep is withheld (`redirect_vlt_artifact_unverifiable`, see the tag table). **Heal**: stale installed copies of Socket-owned nodes are removed so the next `vlt install` extracts the patched bytes, and `rollback` / `remove` do the same for the registry bytes (`--no-vlt-install-cleanup` keeps them; optional dependencies' copies are always kept); `redirect_vlt_reinstall_required` says what happened and what to run. The same-run `--vex` never attests a vlt package whose installed copy is stale or unchecked, whose lock a vlt release may ignore (`redirect_vlt_lockfile_version_missing`, `redirect_vlt_old_lockfile_ignored`, `redirect_vlt_scalar_registry_ignored`), or which also resolves from a non-default registry (`redirect_vlt_custom_registry_skipped`). `vlt.json` or vlt install state without `vlt-lock.json` warns `redirect_vlt_no_lockfile` instead of `redirect_npm_no_lockfile`. `rollback` / `remove` restore each hosted node's slots [2] and [3] from the npm registry, following the lock's own slot-[3] convention (see "Hosted unwind coverage"). Tested releases: `docs/testing/vlt-compatibility.md`.
 
-**Takeover reconciliation (every hosted ecosystem, v5.0)**: vendoring over a hosted pin (`vendor`, `scan --mode vendored`, `get --mode vendored`) first RESTORES that purl's lock entries to their default upstream registry entry — the same restore `rollback` runs (core `patch::redirect::upstream::restore_upstream`; see "Hosted unwind coverage"), over the hosted pins lockfile discovery finds (v5 keeps no hosted ledger) — and then vendors, so the vendor ledger records the PRISTINE registry entry as its wiring `original` and `vendor --revert` lands back on upstream registry state, never on hosted. The run that takes over records a `vendor_takeover_reverted_redirect` advisory event (`skipped` action beside the purl's genuine outcome; detail `<purl> was hosted; restored its upstream registry entry (<files>) before vendoring (mode takeover)`; the human path prints `Warning: …`), plus any advisory the restore raised (`npm_allow_remote_left`, …). `--dry-run` resolves the same restore without writing (registry lookups included): a pin that would restore reports `vendor_would_revert_redirect`, and one that would be refused surfaces in the preview with the wet run's `redirect_revert_failed` code and detail (for bun, whose hosted rewrite replaces the entry's `name@version` spec, the preview first runs the Bun vendored preflight described below and then stops at the advisory instead of reading the still-hosted lock — a lock the vendored backend would refuse is previewed as the wet run's `failed <code>`, never as `vendor_would_revert_redirect`). A purl whose upstream entry cannot be restored — `--offline`, a registry that does not answer, a lock the restore refuses (see "Hosted unwind coverage"; a hosted binary `bun.lockb` pin IS restored for the takeover — its npm registry record is rebuilt natively — while `rollback` / `remove` refuse it) — fails `redirect_revert_failed` with the detail `cannot vendor over the live hosted pin: cannot restore <purl> to its upstream registry entry: <why>; restore it from version control instead (`git checkout -- <files>`)` (exit 1 / `partial_failure`, nothing vendored for it, the hosted wiring left in place). The cargo backend's `hosted_redirect_live` refusal backstops a crate whose hosted residue is still in place when it is reached; its detail names `socket-patch rollback` and `git checkout -- Cargo.toml Cargo.lock`. **Bun vendored preflight before the takeover**: `vendor` — like `scan` / `get --mode vendored`, whose pre-download preflight runs earlier — checks `bun.lock` / `bun.lockb` with the shared Bun vendored preflight BEFORE the upstream restore, so a hosted purl on a lock the vendored backend refuses (a pre-version-2 `workspace:` lock → `vendor_bun_workspace_unsupported`; a malformed or unsupported binary lock → `vendor_bun_lockb_invalid`; an unsupported text-lock version → its code) is reported `failed <code>` with the hosted wiring and active Bun lock byte-untouched (exit 1 / `partial_failure`): the package stays hosted-patched instead of being un-hosted and then refused. `vendor --dry-run` previews that same `failed` code (exit-code parity with the wet run, nothing written) instead of promising `vendor_would_revert_redirect`. Pinned by `tests/in_process_vendor_bun_takeover.rs` and, against real Bun, `tests/mode_migration_bun.rs`. The npm package-lock backend's lock gate gets the same placement: a hosted pin in a project whose `npm-shrinkwrap.json` / `package-lock.json` is not a v2/v3 lock (npm 6's lockfileVersion 1) is refused `failed vendor_lockfile_version_unsupported` BEFORE the restore, in `vendor`, `scan --mode vendored` and `get --mode vendored` alike, so the package stays hosted-patched; the vendored dry-run preview lists every npm purl of such a project as `would_refuse` with that code. Pinned by `tests/in_process_vendor_npm_v1_takeover.rs`. Hosted → vendored and vendored → hosted (`redirect_takeover_reverted_vendored` in `redirect.warnings[]`) both work in place on the locks the target mode accepts. **Removed in v5.0**: the run-level `vendor_supersedes_redirect` warning and its reconcile of the redirect ledger (a live lock that already proved vendored won over a stale hosted ledger record) — once the lock routes a package to `.socket/vendor/`, no hosted state is left to go stale. Which way the live lock points is decided by the same lockfile discovery rules `vex` gates attestations on (see "Manifest-less VEX (lockfile discovery)"), for `redirect_supersedes_vendored` and `hosted_wiring_retained` alike.
+**Takeover reconciliation (every hosted ecosystem, v5.0)**: vendoring over a hosted pin (`vendor`, `scan --mode vendored`, `get --mode vendored`) first RESTORES that purl's lock entries to their default upstream registry entry — the same restore `rollback` runs (core `patch::redirect::upstream::restore_upstream`; see "Hosted unwind coverage"), over the hosted pins lockfile discovery finds (v5 keeps no hosted ledger) — and then vendors, so the vendor ledger records the PRISTINE registry entry as its wiring `original` and `vendor --revert` lands back on upstream registry state, never on hosted. The run that takes over records a `vendor_takeover_reverted_redirect` advisory event (`skipped` action beside the purl's genuine outcome; detail `<purl> was hosted; restored its upstream registry entry (<files>) before vendoring (mode takeover)`; the human path prints `Warning: …`), plus any advisory the restore raised (`npm_allow_remote_left`, …). `--dry-run` resolves the same restore without writing (registry lookups included): a pin that would restore reports `vendor_would_revert_redirect`, and one that would be refused surfaces in the preview with the wet run's `redirect_revert_failed` code and detail (for bun, whose hosted rewrite replaces the entry's `name@version` spec, the preview first runs the Bun vendored preflight described below and then stops at the advisory instead of reading the still-hosted lock — a lock the vendored backend would refuse is previewed as the wet run's `failed <code>`, never as `vendor_would_revert_redirect`). A purl whose upstream entry cannot be restored — `--offline`, a registry that does not answer, a lock the restore refuses (see "Hosted unwind coverage"; a hosted binary `bun.lockb` pin IS restored for the takeover — its npm registry record is rebuilt natively — while `rollback` / `remove` refuse it) — fails `redirect_revert_failed` with the detail `cannot vendor over the live hosted pin: cannot restore <purl> to its upstream registry entry: <why>; restore it from version control instead (`git checkout -- <files>`)` (exit 1 / `partial_failure`, nothing vendored for it, the hosted wiring left in place). The cargo backend's `hosted_redirect_live` refusal backstops a crate whose hosted residue is still in place when it is reached; its detail names `socket-patch rollback` and `git checkout -- Cargo.toml Cargo.lock`. **Bun vendored preflight before the takeover**: `vendor` — like `scan` / `get --mode vendored`, whose pre-download preflight runs earlier — checks `bun.lock` / `bun.lockb` with the shared Bun vendored preflight BEFORE the upstream restore, so a hosted purl on a lock the vendored backend refuses (a pre-version-2 `workspace:` lock → `vendor_bun_workspace_unsupported`; a malformed or unsupported binary lock → `vendor_bun_lockb_invalid`; an unsupported text-lock version → its code) is reported `failed <code>` with the hosted wiring and active Bun lock byte-untouched (exit 1 / `partial_failure`): the package stays hosted-patched instead of being un-hosted and then refused. `vendor --dry-run` previews that same `failed` code (exit-code parity with the wet run, nothing written) instead of promising `vendor_would_revert_redirect`. Pinned by `tests/in_process_vendor_bun_takeover.rs` and, against real Bun, `tests/mode_migration_bun.rs`. The npm package-lock backend's lock gate gets the same placement: a hosted pin in a project whose `npm-shrinkwrap.json` / `package-lock.json` is not a v2/v3 lock (npm 6's lockfileVersion 1) is refused `failed vendor_lockfile_version_unsupported` BEFORE the restore, in `vendor`, `scan --mode vendored` and `get --mode vendored` alike, so the package stays hosted-patched; the vendored dry-run preview lists every npm purl of such a project as `would_refuse` with that code. Pinned by `tests/in_process_vendor_npm_v1_takeover.rs`. Hosted → vendored and vendored → hosted (`redirect_takeover_reverted_vendored` in `redirect.warnings[]`) both work in place on the locks the target mode accepts. **Removed in v5.0**: the run-level `vendor_supersedes_redirect` warning and its reconcile of the redirect ledger (a live lock that already proved vendored won over a stale hosted ledger record) — once the lock routes a package to `.socket/vendor/`, no hosted state is left to go stale. Which way the live lock points is decided by the same lockfile discovery rules `vex` gates attestations on (see "Manifest-less VEX (lockfile discovery)"), for `redirect_supersedes_vendored` and `hosted_wiring_retained` alike. **Gem preflight before the takeover**: `scan` / `get --mode vendored` ask the gem vendored backend's own refusals BEFORE the upstream restore — the manifest gate (`gemfile_not_loaded`: a `gems.rb` twin or a `BUNDLE_GEMFILE`-configured manifest) and the Gemfile declaration gate evaluated on the Gemfile and Gemfile.lock text the restore would leave (`gemfile_declaration_not_editable`: a declaration inside a `group` / `platforms` / conditional block, a parenthesized or duplicate declaration, …) — so a hosted gem vendored mode cannot wire is reported `failed <code>` with the hosted `Gemfile` / `Gemfile.lock` byte-untouched (exit 1 / `partial_failure`), and their `--dry-run` preview reports that gem as `would_refuse` with the same `errorCode` (exit 0, like the Bun / vlt `would_refuse` rows), never `would_vendor`. Pinned against real Bundler by `tests/e2e_redirect_gem_build.rs`. Hosted → vendored and vendored → hosted (`redirect_takeover_reverted_vendored` in `redirect.warnings[]`) both work in place on the locks the target mode accepts. **Removed in v5.0**: the run-level `vendor_supersedes_redirect` warning and its reconcile of the redirect ledger (a live lock that already proved vendored won over a stale hosted ledger record) — once the lock routes a package to `.socket/vendor/`, no hosted state is left to go stale. Which way the live lock points is decided by the same lockfile discovery rules `vex` gates attestations on (see "Manifest-less VEX (lockfile discovery)"), for `redirect_supersedes_vendored` and `hosted_wiring_retained` alike.
 
 ### Scan modes (v5.0)
 

crates/socket-patch-cli/src/commands/get.rs

@@ -1742,7 +1742,11 @@ type LockRefusals = HashMap<String, (&'static str, String)>;
 /// refusal and the ledger's idempotency skip, which take precedence in the
 /// fetch loop. A purl the lockfiles pin hosted is left to the vendor loop:
 /// its takeover restores the upstream lock entry first, and the restore
-/// rewrites the very text the gates read.
+/// rewrites the very text the gates read. The one exception is a hosted
+/// gem the takeover would refuse ([`gem_takeover_refusals_for`]), which
+/// is refused here with the takeover's code instead of after its fetch.
+///
+/// [`gem_takeover_refusals_for`]: crate::commands::vendor::gem_takeover_refusals_for
 ///
 /// Only a package the vendor loop would hand to its backend is refused
 /// here (see [`crate::commands::vendor::lock_refusals_reaching_backend`]):
@@ -1759,44 +1763,65 @@ async fn lock_text_refusals_for(
     prior: Option<&crate::ecosystem_dispatch::NpmCrawlSnapshot>,
 ) -> LockRefusals {
     let cwd = params.cwd.as_path();
-    let claimed: Vec<String> = socket_patch_core::patch::redirect::upstream::HostedPin::all(
+    let origins: Vec<String> = params
+        .patch_server_url
+        .iter()
+        .filter(|url| !url.trim().is_empty())
+        .cloned()
+        .collect();
+    let pins = socket_patch_core::patch::redirect::upstream::HostedPin::all(
         &socket_patch_core::vex::discover_patched_refs_with(
             cwd,
             &socket_patch_core::vex::DiscoverOptions {
-                patch_server_origins: params
-                    .patch_server_url
-                    .iter()
-                    .filter(|url| !url.trim().is_empty())
-                    .cloned()
-                    .collect(),
+                patch_server_origins: origins.clone(),
             },
         )
         .await,
-    )
-    .into_iter()
-    .map(|pin| canonical_purl(&pin.purl))
-    .collect();
-    let candidates: Vec<(&str, &str)> = selected
+    );
+    let claimed: Vec<String> = pins.iter().map(|pin| canonical_purl(&pin.purl)).collect();
+    let fetchable: Vec<&PatchSearchResult> = selected
         .iter()
         .filter(|sr| bun_refusal.filter(|r| r.applies_to(&sr.purl)).is_none())
         .filter(|sr| {
             detached_ledger_record(RecordStore::Ledger(&ledger.entries), &sr.purl, &sr.uuid)
                 .is_none()
         })
+        .collect();
+    let candidates: Vec<(&str, &str)> = fetchable
+        .iter()
         .filter(|sr| !claimed.contains(&canonical_purl(&sr.purl)))
         .map(|sr| (sr.purl.as_str(), sr.uuid.as_str()))
         .collect();
     let refused = socket_patch_core::vendor::lock_text_refusals(cwd, &candidates).await;
     let options = params.crawler_options();
-    crate::commands::vendor::lock_refusals_reaching_backend(
-        cwd,
-        refused,
-        &ledger.entries,
-        |purls| async move {
-            crate::commands::vendor::installed_purls(&options, &purls, prior).await
-        },
-    )
-    .await
+    let mut refusals =
+        crate::commands::vendor::lock_refusals_reaching_backend(
+            cwd,
+            refused,
+            &ledger.entries,
+            |purls| async move {
+                crate::commands::vendor::installed_purls(&options, &purls, prior).await
+            },
+        )
+        .await;
+    // A hosted gem the takeover will refuse (#775) is refused here too, so
+    // its view is never fetched for a package the run cannot vendor. The
+    // download phase only runs online (`--offline` refuses `get` and `scan`
+    // before it), so the dry-run restore may resolve the registry entry.
+    refusals.extend(
+        crate::commands::vendor::gem_takeover_refusals_for(
+            cwd,
+            fetchable
+                .iter()
+                .filter(|sr| claimed.contains(&canonical_purl(&sr.purl)))
+                .map(|sr| sr.purl.as_str()),
+            &pins,
+            false,
+            origins,
+        )
+        .await,
+    );
+    refusals
 }
 
 /// The record a detached ledger entry already carries for `purl` at
@@ -3643,7 +3668,12 @@ async fn run_get_vendored(
     // Dry run: ledger-classification preview only (scan's posture) — no
     // download, no vendor step, no writes.
     if args.common.dry_run {
-        let preview = super::scan::preview_vendor_json(&args.common.cwd, selected).await;
+        let takeover = super::vendor::gem_takeover_preview_refusals(
+            &args.common,
+            selected.iter().map(|p| p.purl.as_str()),
+        )
+        .await;
+        let preview = super::scan::preview_vendor_json(&args.common.cwd, selected, &takeover).await;
         if args.common.json {
             let mut result = serde_json::json!({
                 "status": "success",

crates/socket-patch-cli/src/commands/scan/mod.rs

@@ -2923,12 +2923,13 @@ async fn run_scan(
     };
 
     // The rollout plan (§5.2): deferred NEW rows leave the selection here.
-    // Vendored eligibility is the wet run's Bun / vlt preflight; the agent
-    // partition above already removed what cannot land in place.
+    // Vendored eligibility is the wet run's Bun / vlt / npm-lock / gem
+    // takeover preflight; the agent partition above already removed what
+    // cannot land in place.
     let selected = if report_only {
         selected
     } else if vendor {
-        let refused = vendor_flow::preflight_refused_purls(&args.common.cwd, &selected).await;
+        let refused = vendor_flow::preflight_refused_purls(&args.common, &selected).await;
         stage.plan(&rows, |r| !refused.contains(&r.writer.purl));
         let deferred = stage.deferred_keys();
         selected
@@ -3047,7 +3048,12 @@ async fn run_scan(
             // rendered as `[would-refuse]` lines so a preview never
             // advertises vendoring the wet run would refuse.
             let preview = if vendor {
-                Some(preview_vendor_json(&args.common.cwd, &selected).await)
+                let takeover = crate::commands::vendor::gem_takeover_preview_refusals(
+                    &args.common,
+                    selected.iter().map(|p| p.purl.as_str()),
+                )
+                .await;
+                Some(preview_vendor_json(&args.common.cwd, &selected, &takeover).await)
             } else {
                 None
             };