Skip to content

feat(supervisor): set up sandbox netns via rtnetlink instead of ip/nsenter - #3285

Open
akram wants to merge 1 commit into
NVIDIA:mainfrom
akram:refactor/netns-rtnetlink
Open

feat(supervisor): set up sandbox netns via rtnetlink instead of ip/nsenter#3285
akram wants to merge 1 commit into
NVIDIA:mainfrom
akram:refactor/netns-rtnetlink

Conversation

@akram

@akram akram commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Summary

The privileged supervisor builds the sandbox network namespace by shelling out to ip (32 call sites) and, for in-namespace operations, nsenter — both resolved from the workload image. A bare docker.io/library/alpine:3.22 image ships only busybox ip (no netns subcommand), so proxy-mode setup fails with Network namespace creation failed … iproute2 is installed.

This change programs the namespace entirely through kernel interfaces, so the supervisor needs no network tooling from the workload image:

  • veth pair, addresses, link up/down, routes are programmed over route netlink (rtnetlink).
  • The namespace is created with unshare(CLONE_NEWNET) on a short-lived thread and bind-mounted onto netns_path via mount(2) (not ip netns add), so it persists and stays reachable for the still-nsenter-based nft path.
  • In-namespace operations run on a dedicated thread that setns() into the namespace, instead of spawning nsenter.

rtnetlink is async; it is confined to the new netns/netlink.rs module and driven from the synchronous create() via a local current_thread runtime on a dedicated OS threadcreate() runs inside a tokio runtime on the Podman driver, so building the runtime on the caller thread would panic (“Cannot start a runtime from within a runtime”).

Scope is the ip/nsenter namespace-setup path only. The nft bypass-rule path (still nsenter-based) and dmesg→NFLOG are separate follow-ups, so nsenter and the nft helpers are intentionally left in place.

Related Issue

Refs #3281 — networking (ip/nsenter) subset of #2750 (make the privileged supervisor independent of workload-image tooling). Unblocks the bare-Alpine default in #3116.

Changes

  • Add crates/openshell-supervisor-process/src/netns/netlink.rs: FD-owned namespace creation, host-side veth setup, setns-thread in-namespace configuration, veth/namespace teardown, and route replace/dump — all over rtnetlink + unshare/setns/mount.
  • Rewire NetworkNamespace::create(), Drop, install_transparent_tcp_rules, and validate_synthetic_pool_routes onto the new module; first_route_overlap now operates on parsed route prefixes.
  • Remove the dead run_ip / run_ip_netns / run_ip_netns_output helpers and IP_SEARCH_PATHS; drop the iproute2 hint from the proxy-mode error message.
  • Add rtnetlink, netlink-packet-route, and futures-util under the crate’s cfg(target_os = "linux") dependencies.

Testing

  • Built the supervisor musl-static (x86_64-unknown-linux-musl) with the new dependencies.
  • End-to-end on a rootless-Podman host: a bare docker.io/library/alpine:3.22 sandbox starts in proxy mode with the network namespace, veth pair, addresses, and default route established via netlink and no ip/nsenter process spawned; the FD-owned namespace bind mount is present, and teardown removes the veth and namespace. The sandbox reaches Ready/healthy.
  • Root-only integration tests for the netlink primitives are included (gated with #[ignore]; require CAP_NET_ADMIN).
  • No behavior change for images that already ship iproute2/nftables.

Checklist

  • Conventional commit, signed off (DCO)
  • Scoped to the issue; nft/dmesg paths left for follow-ups
  • Supervisor stays musl-static (dependencies are pure Rust)
  • No behavior change for images shipping iproute2/nftables
  • Maintainer review

…enter

Replace the privileged supervisor's `ip` (32 call sites) and the
`ip`-in-namespace `nsenter` usage with in-process route netlink
(rtnetlink) plus unshare/setns/mount syscalls, so proxy-mode sandboxes no
longer need iproute2/nsenter from the workload image and a bare Alpine
image works.

- New netns/netlink.rs programs the veth pair, addresses, routes, and
  teardown over rtnetlink. The namespace is created with
  unshare(CLONE_NEWNET) and bind-mounted onto netns_path via mount(2)
  (not `ip netns add`), so it stays reachable for the still-nsenter-based
  nft path, which is converted separately.
- rtnetlink is async; it is driven from the synchronous create() via a
  local current-thread runtime on a dedicated OS thread. create() runs
  inside a tokio runtime on the podman driver, so building the runtime on
  the caller thread would panic ("runtime within a runtime");
  namespace-scoped operations run on a setns thread.
- Remove run_ip/run_ip_netns/run_ip_netns_output and IP_SEARCH_PATHS, and
  drop the iproute2 hint from the proxy-mode error. nsenter and the nft
  helpers are unchanged (the nft path is a separate follow-up).

Refs NVIDIA#3281

Signed-off-by: Akram <akram.benaissi@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 11, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@akram
akram marked this pull request as ready for review September 11, 2026 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant