Skip to content

Commit a176aa7

Browse files
test(e2e): demote the cargo production legs to canary status — free cargo tier is unpublished (#235)
* test(e2e): demote the cargo production legs to canary status — free cargo tier is unpublished Production deleted both pinned traitobject patches between 2026-08-27 and 2026-08-28 (0.1.1/cf2e6f58 for the hosted/vendored suites, 0.0.1/b15f2b7f for the safety round-trip), and now publishes no free-tier patch for ANY cargo crate — probes of every RUSTSEC-advisoried crate and the top 1000 crates.io packages all return empty, while the npm/pypi/gem pins remain live. With nothing to pin, the cargo install proofs cannot run; this has kept hosted-e2e and the three e2e_safety_cargo_build legs red on main since 2026-08-28. Move cargo onto the UNPUBLISHED_ECOSYSTEMS watchlist in both production suites, exactly as maven/nuget/composer are handled: the canaries keep probing production each run and, under the CANARY_STRICT knobs, nag when a free cargo patch appears again so the install proofs can be restored. Retire traitobject_real_socket_patch_round_trip (its oracle was that specific patch's compile_error!) and the cargo-only helpers nothing else references. Docs get the corrected catalog/coverage tables plus a dated demotion note; re-promotion is the existing withdrawn-patch procedure with this commit's history as the restore template. Verified: safety suite 5/5; hosted suite 15/15 live against production under SOCKET_PATCH_HOSTED_E2E_STRICT=1; vendored suite 11/12 (the one red is pre-existing server-side gem-catalog drift, untouched by this change); CI's clippy invocation clean; no traitobject reference remains outside get.rs's mocked unit tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): regenerate the npm wrapper lock with CI's npm major release-readiness regenerates the lock with the runner's npm 10, which does not write the `libc` platform arrays npm >= 11 emits — so the npm-11-shaped lock from #233 makes 'version-sync.sh 4.0.0 is not a no-op' fail on every PR and on main. The check's npm is the effective canon; note the npm-major dependence for whoever next refreshes the lock locally. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ci): pin the npm wrapper lock refresh to npm@10 via npx The release-readiness gate re-runs version-sync.sh and compares the regenerated npm/socket-patch/package-lock.json byte-for-byte, so the lock's canonical shape is defined by whatever npm regenerates it. npm 11 adds libc arrays that npm 10 omits, which is how the #233 refresh (made locally with npm 11) broke the gate under CI's npm 10 — and the same drift would recur in reverse the day the runner image jumps to npm 11. Pinning the refresh via npx makes the gate independent of both the runner default and the developer's local npm; bumping the pin now takes a deliberate commit that refreshes the lock alongside it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent d5e1815 commit a176aa7

7 files changed

Lines changed: 109 additions & 597 deletions

File tree

crates/socket-patch-cli/tests/e2e_hosted_production.rs

Lines changed: 32 additions & 131 deletions
Original file line numberDiff line numberDiff line change
@@ -35,19 +35,21 @@
3535
//! |-----------|------|------------|----------|
3636
//! | npm | `pkg:npm/minimist@1.2.2` | `80630680-4da6-45f9-bba8-b888e0ffd58c` | GHSA-xvch-5gv4-984h (CVE-2021-44906) |
3737
//! | PyPI | `pkg:pypi/urllib3@1.26.18` | *any of three* (see [`PYPI_UUIDS`]) | GHSA-gm62-xv2j-4w53 &co |
38-
//! | Cargo | `pkg:cargo/traitobject@0.1.1` | `cf2e6f58-d9fa-4096-9151-c34afa717f89` | GHSA-pp8r-vv2j-9j5v |
3938
//! | gem | `pkg:gem/activestorage@6.0.3` | *any of* [`GEM_UUIDS`] (five today) | GHSA-m42x-37p3-fv5w (CVE-2020-8162), GHSA-w749-p3v6-hccq (CVE-2022-21831), GHSA-9xrj-h377-fr87 (CVE-2026-33195), GHSA-r4mg-4433-c7g3 (CVE-2025-24293), GHSA-xr9x-r78c-5hrm (CVE-2026-66066) |
4039
//!
4140
//! `docs/testing/hosted-production-e2e.md` explains how these were chosen and
4241
//! how to re-pick one if it is ever withdrawn.
4342
//!
4443
//! # Ecosystems with no coverage, and why
4544
//!
46-
//! * **maven / nuget / composer** — hosted mode is implemented and documented
47-
//! for all three, but production currently publishes **zero** free-tier
48-
//! patches for them, so there is nothing real to redirect to. Rather than
49-
//! silently skipping, [`canary_unpublished_ecosystems`] probes production
50-
//! every run and tells us the moment that changes.
45+
//! * **cargo / maven / nuget / composer** — hosted mode is implemented and
46+
//! documented for all four, but production currently publishes **zero**
47+
//! free-tier patches for them, so there is nothing real to redirect to.
48+
//! Rather than silently skipping, [`canary_unpublished_ecosystems`] probes
49+
//! production every run and tells us the moment that changes. cargo carried
50+
//! a full sparse-registry install proof until 2026-09-01: production's free
51+
//! cargo tier emptied on 2026-08-28, so the leg was demoted to the canary
52+
//! (`docs/testing/hosted-production-e2e.md` says how to re-promote it).
5153
//! * **golang** — hosted mode is supported for free-tier references carrying
5254
//! a `goproxy` override (`docs/design/golang-hosted.md`), but production
5355
//! publishes no golang hosted modules yet. Covered as a shape guard that
@@ -58,11 +60,11 @@
5860
//!
5961
//! Toolchains (each leg soft-skips if its own toolchain is absent, unless
6062
//! `SOCKET_PATCH_HOSTED_E2E_STRICT=1`): `npm`, `pnpm`, `yarn` (classic),
61-
//! `corepack` (berry), `bun`, `uv`, `cargo`, `ruby` + `bundle`, `go`.
63+
//! `corepack` (berry), `bun`, `uv`, `ruby` + `bundle`, `go`.
6264
//!
6365
//! Network egress to: `patches-api.socket.dev`, `patch.socket.dev`,
6466
//! `registry.npmjs.org`, `pypi.org`, `files.pythonhosted.org`,
65-
//! `static.crates.io`, `index.crates.io`, `rubygems.org`.
67+
//! `rubygems.org`.
6668
//!
6769
//! No API token is used or needed — the suite deliberately runs against the
6870
//! **free public proxy**, which is the surface every unauthenticated user
@@ -118,16 +120,6 @@ const PYPI_UUIDS: &[&str] = &[
118120
"e828efa5-5c6d-43f3-9909-03f5ac232b98",
119121
];
120122

121-
const CARGO_PURL: &str = "pkg:cargo/traitobject@0.1.1";
122-
const CARGO_NAME: &str = "traitobject";
123-
const CARGO_VERSION: &str = "0.1.1";
124-
const CARGO_UUID: &str = "cf2e6f58-d9fa-4096-9151-c34afa717f89";
125-
/// The traitobject patch annotates `src/lib.rs` with its advisory ID (the
126-
/// crate is unmaintained; the patch documents that and fixes deprecations).
127-
/// Cargo crates are not rewritten with the `// Socket Community Patch` header
128-
/// that npm/PyPI artifacts carry, so this is the marker to look for.
129-
const CARGO_MARKER: &str = "GHSA-pp8r-vv2j-9j5v";
130-
131123
/// The gem pin is deliberately UNQUALIFIED. Production publishes the purl as
132124
/// `pkg:gem/activestorage@6.0.3?platform=ruby`, but nothing client-side
133125
/// strips qualifiers — the SERVER normalizes both spellings to the same
@@ -182,6 +174,19 @@ const PATCH_MARKER: &str = "Socket Community Patch";
182174
/// patches to exercise it with. [`canary_unpublished_ecosystems`] watches
183175
/// these so coverage can be extended the moment one lights up.
184176
const UNPUBLISHED_ECOSYSTEMS: &[(&str, &[&str])] = &[
177+
// cargo joined this list on 2026-09-01: production deleted its last free
178+
// cargo patches on 2026-08-28, retiring the pinned sparse-registry
179+
// install proof this suite used to carry. Re-promotion procedure:
180+
// docs/testing/hosted-production-e2e.md.
181+
(
182+
"cargo",
183+
&[
184+
"pkg:cargo/openssl",
185+
"pkg:cargo/tokio",
186+
"pkg:cargo/hyper",
187+
"pkg:cargo/smallvec",
188+
],
189+
),
185190
(
186191
"maven",
187192
&[
@@ -729,7 +734,6 @@ async fn preflight_required_patches_are_published() {
729734
let required: Vec<(&str, Vec<&str>)> = vec![
730735
(NPM_PURL, vec![NPM_UUID]),
731736
(PYPI_PURL, PYPI_UUIDS.to_vec()),
732-
(CARGO_PURL, vec![CARGO_UUID]),
733737
(GEM_PURL, GEM_UUIDS.to_vec()),
734738
];
735739

@@ -783,7 +787,7 @@ async fn canary_patches_name_advisories_so_merge_state_is_inferable() {
783787
let mut failures: Vec<String> = Vec::new();
784788
let mut coverage_seen: Vec<(String, String, usize)> = Vec::new();
785789

786-
let canary_purls = vec![NPM_PURL, PYPI_PURL, CARGO_PURL, GEM_PURL];
790+
let canary_purls = vec![NPM_PURL, PYPI_PURL, GEM_PURL];
787791
for purl in canary_purls {
788792
match published_patch_advisory_counts(purl).await {
789793
Err(e) => failures.push(format!("{purl}: production probe failed: {e}")),
@@ -1531,111 +1535,6 @@ fn pypi_uv_lock_hosted_install_proof() {
15311535
);
15321536
}
15331537

1534-
// ===========================================================================
1535-
// Cargo — per-patch sparse registry
1536-
// ===========================================================================
1537-
1538-
#[test]
1539-
#[ignore = "live production API + real crates.io. Run with --ignored."]
1540-
fn cargo_hosted_install_proof() {
1541-
const LEG: &str = "cargo_hosted_install_proof";
1542-
if !has_command("cargo") {
1543-
soft_skip!(LEG, "`cargo` not on PATH");
1544-
}
1545-
let tmp = tempfile::tempdir().expect("tempdir");
1546-
let proj = tmp.path().join("proj");
1547-
std::fs::create_dir_all(proj.join("src")).expect("mkdir src");
1548-
let home = tmp.path().join("cargo-home").display().to_string();
1549-
let env = [("CARGO_HOME", home.as_str())];
1550-
1551-
std::fs::write(
1552-
proj.join("Cargo.toml"),
1553-
format!(
1554-
"[package]\nname = \"hosted-e2e\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n\
1555-
[dependencies]\n{CARGO_NAME} = \"={CARGO_VERSION}\"\n"
1556-
),
1557-
)
1558-
.expect("write Cargo.toml");
1559-
std::fs::write(proj.join("src").join("main.rs"), "fn main() {}\n").expect("write main.rs");
1560-
1561-
let fetch = tool(&proj, "cargo", &["fetch"], &env);
1562-
if !ok(&fetch) {
1563-
soft_skip!(LEG, "upstream `cargo fetch` failed:\n{}", dump(&fetch));
1564-
}
1565-
let pristine_lock = read(&proj.join("Cargo.lock"));
1566-
assert!(
1567-
pristine_lock.contains("registry+https://github.com/rust-lang/crates.io-index"),
1568-
"{LEG}: pristine Cargo.lock does not resolve {CARGO_NAME} from \
1569-
crates.io — fixture setup is wrong:\n{pristine_lock}"
1570-
);
1571-
1572-
let env_json = scan_hosted(&proj, &[]);
1573-
assert_redirected(&env_json, "Cargo.lock");
1574-
1575-
let lock = read(&proj.join("Cargo.lock"));
1576-
assert_hosted_pin(&lock, &[CARGO_UUID], LEG);
1577-
let config = read(&proj.join(".cargo").join("config.toml"));
1578-
assert!(
1579-
config.contains(&format!(
1580-
"sparse+https://{PATCH_HOST}/patch-registry/cargo/"
1581-
)),
1582-
"{LEG}: .cargo/config.toml declares no Socket sparse registry:\n{config}"
1583-
);
1584-
let manifest = read(&proj.join("Cargo.toml"));
1585-
assert!(
1586-
manifest.contains(&format!("socket-patch-{CARGO_UUID}")),
1587-
"{LEG}: Cargo.toml does not route {CARGO_NAME} at the per-patch \
1588-
registry:\n{manifest}"
1589-
);
1590-
1591-
// Proof: fetch again with a cold CARGO_HOME so cargo must reach the Socket
1592-
// sparse index, download the crate, and verify the checksum in the lock.
1593-
let cold = tmp.path().join("cargo-home-cold").display().to_string();
1594-
let cold_env = [("CARGO_HOME", cold.as_str())];
1595-
let refetch = tool(&proj, "cargo", &["fetch"], &cold_env);
1596-
assert!(
1597-
ok(&refetch),
1598-
"{LEG}: `cargo fetch` from the Socket sparse registry failed — cargo \
1599-
could not reach the index, download the crate, or verify its \
1600-
checksum:\n{}",
1601-
dump(&refetch)
1602-
);
1603-
1604-
// The extracted source must be the patched crate, not the crates.io one.
1605-
let src_root = Path::new(&cold).join("registry").join("src");
1606-
let mut found = None;
1607-
if let Ok(hosts) = std::fs::read_dir(&src_root) {
1608-
for host in hosts.flatten() {
1609-
let candidate = host
1610-
.path()
1611-
.join(format!("{CARGO_NAME}-{CARGO_VERSION}"))
1612-
.join("src")
1613-
.join("lib.rs");
1614-
if candidate.exists() {
1615-
found = Some(candidate);
1616-
break;
1617-
}
1618-
}
1619-
}
1620-
let lib_rs = found.unwrap_or_else(|| {
1621-
panic!("{LEG}: no extracted {CARGO_NAME}-{CARGO_VERSION}/src/lib.rs under {src_root:?}")
1622-
});
1623-
assert!(
1624-
lib_rs
1625-
.parent()
1626-
.and_then(|p| p.parent())
1627-
.and_then(|p| p.parent())
1628-
.and_then(|p| p.file_name())
1629-
.map(|n| n.to_string_lossy().contains(PATCH_HOST))
1630-
.unwrap_or(false),
1631-
"{LEG}: {CARGO_NAME} was extracted from a non-Socket registry dir \
1632-
({}) — cargo served it from the crates.io cache instead of the \
1633-
redirect",
1634-
lib_rs.display()
1635-
);
1636-
assert_patched(&lib_rs, CARGO_MARKER, LEG);
1637-
}
1638-
16391538
// ===========================================================================
16401539
// RubyGems — full hosted install proof
16411540
// ===========================================================================
@@ -1960,10 +1859,12 @@ fn deno_hosted_is_unsupported() {
19601859
// Canary — ecosystems whose hosted support has nothing to test against
19611860
// ===========================================================================
19621861

1963-
/// maven, nuget and composer all implement hosted mode, but production
1862+
/// cargo, maven, nuget and composer all implement hosted mode, but production
19641863
/// publishes no free-tier patches for them, so there is no honest end-to-end
1965-
/// leg to write. This probes production every run and reports the moment that
1966-
/// changes, so coverage can be extended deliberately rather than by accident.
1864+
/// leg to write. (cargo used to have one — the pinned sparse-registry install
1865+
/// proof retired 2026-09-01 when production's free cargo tier emptied.) This
1866+
/// probes production every run and reports the moment that changes, so
1867+
/// coverage can be extended deliberately rather than by accident.
19671868
///
19681869
/// It deliberately does NOT fail when patches appear: production publishing a
19691870
/// new patch is not a socket-patch regression, and a required check must not
@@ -1996,9 +1897,9 @@ async fn canary_unpublished_ecosystems() {
19961897

19971898
if newly_published.is_empty() {
19981899
println!(
1999-
"canary_unpublished_ecosystems: maven / nuget / composer still have \
2000-
no free-tier published patches — their hosted-mode legs remain \
2001-
untestable end-to-end against production."
1900+
"canary_unpublished_ecosystems: cargo / maven / nuget / composer \
1901+
still have no free-tier published patches — their hosted-mode legs \
1902+
remain untestable end-to-end against production."
20021903
);
20031904
return;
20041905
}

0 commit comments

Comments
 (0)