refactor(config): enforce gateway schema version 2

[jhjaggars]

Summary

Cut the gateway configuration surface over to strict schema version 2 and complete the compute-driver configuration normalization tracked by #2792. The schema gives each setting one canonical name and owner, replaces sentinel values with typed optional values, centralizes shared driver models, and preserves runtime and persisted-state guarantees at explicit compatibility boundaries.

Related Issue

Closes #2792

Changes

• Require explicit [openshell] version = 2; reject omitted, version 1, and unsupported future schemas with migration-focused diagnostics.
• Replace plural driver selection with compute_driver, --compute-driver, and OPENSHELL_COMPUTE_DRIVER; reject legacy compute_drivers, --drivers, and OPENSHELL_DRIVERS inputs, including an explicitly empty legacy environment variable.
• Remove gateway-to-driver inheritance and require driver-owned values under [openshell.drivers.<name>]; reject non-table driver entries and stale Docker, Podman, and VM aliases.
• Keep the OpenShell 2048-process guard when Docker or Podman PID limits are omitted, while continuing to reject explicit zero.
• Add shared typed image-pull and AppArmor policies with backend-specific translation and early rejection of unsupported combinations.
• Preserve Helm values-file compatibility by translating Kubernetes pull-policy spellings (Always, IfNotPresent, and Never) to schema-v2 values while rejecting unsupported policies during rendering.
• Derive default callback endpoints by runtime topology while retaining explicit per-driver overrides.
• Make guest TLS bundle ownership gateway-wide, validate complete bundles, and inject credentials only into the selected local driver.
• Preserve VM image account metadata and record each persisted overlay's exact UID/GID; migrate older overlays by inspecting their persisted prepared rootfs before using explicit or legacy fallbacks.
• Add Docker and VM corporate-proxy propagation, a Docker-compatible read-only provider-SPIFFE socket projection, explicit VM guest-TCP SPIFFE acknowledgement, and Docker/Podman/Kubernetes AppArmor parity.
• Keep Podman health checks optional without weakening public readiness: a sandbox remains Provisioning with SupervisorNotConnected until the supervisor control session connects.
• Update Helm, packaged defaults, deployment assets, scripts, E2E fixtures, architecture docs, driver READMEs, agent diagnostics, and published migration/reference documentation.

Breaking migration notes

• Existing files must add [openshell] version = 2 and use the singular driver selector.
• Driver settings no longer inherit from [openshell.gateway]; move them into the selected driver table. Keep only guest_tls_ca, guest_tls_cert, and guest_tls_key at gateway scope.
• Rename Docker sandbox_namespace to sandbox_label, Podman sandbox_ssh_socket_path to ssh_socket_path, and VM openshell_endpoint to grpc_endpoint.
• Raw gateway TOML uses image pull policies always, if_not_present, never, or Podman-only newer. Existing Helm values may retain Kubernetes spellings because the chart normalizes them.
• Omit JWT TTL to disable expiry, omit Docker or Podman PID limits to use OpenShell's 2048-process default, and omit Podman health-check intervals to disable backend probes. Explicit zero remains invalid.

The complete migration procedure and schema-v2 examples are in docs/reference/gateway-config.mdx.

Testing

• mise run pre-commit
• mise run test
• mise run ci
• mise run helm:test
• Targeted Docker, Podman, server CLI, Helm, and VM unit tests
• Podman schema-v2 E2E: all applicable suites passed; the host-only keep-id case cannot run because host UID 16446 is absent from the passwd database
• External Podman driver CLI conformance and gateway restart E2E
• Docker/Kubernetes E2E (unavailable: no Docker daemon on this host)
• VM E2E (unavailable: /dev/kvm is absent)

The local Git configuration signs tags globally, so repository-wide test commands used a process-local tag.gpgSign=false override for temporary test repositories. This does not affect product behavior or committed Git configuration.

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated
• User-facing and deployment documentation updated
• Breaking changes and migration steps documented

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[jhjaggars]

Review concern dispositions and completed follow-up

The hard schema-v2 cut remains intentional: OpenShell does not silently accept v1 configuration syntax. Compatibility is preserved at explicit operational boundaries where an upgrade must not break runtime behavior, persisted state, or stable Helm values.

Concern	Final disposition	Resolution
Existing Homebrew and RPM v1 files fail after upgrade	Intentional compatibility break	Keep strict schema-v2 validation and migration-focused startup errors rather than adding a general v1 migration shim.
Docker provider-SPIFFE bind uses unsupported rbind	Fixed	Project the socket's parent directory with Docker's supported read-only bind syntax and assert the exact bind string in tests.
Podman AppArmor capability is silently read as false	Fixed	Deserialize Podman's camelCase apparmorEnabled field and test confined profiles against both available and unavailable capability reports.
Removed OPENSHELL_DRIVERS is silently ignored	Fixed	Reject the variable whenever it is present, including an empty value, and direct users to OPENSHELL_COMPUTE_DRIVER.
Omitted PID limit removes the 2048 fork-bomb guard	Fixed	Omission retains OpenShell's 2048-process default for Docker and Podman; documentation no longer calls this the container runtime default.
Podman readiness can bypass or hang when health checks are omitted	No readiness code change required; documentation fixed	Backend health checks remain optional. Gateway composition keeps the public sandbox phase in Provisioning with SupervisorNotConnected until a supervisor session registers, so omission cannot make a sandbox publicly Ready early.
VM legacy UID 10001 and image-provided accounts are not preserved	Fixed	Store exact UID/GID in a v2 overlay marker, recover unmarked state from the persisted prepared rootfs when possible, use explicit configuration before the legacy fallback, and preserve image-owned account metadata while reconciling IDs.
Kubernetes/Helm pull-policy capitalization is rejected	Fixed at the Helm boundary	Raw schema-v2 TOML stays canonical. Helm accepts stable legacy values Always, IfNotPresent, and Never, translates them to canonical values, and fails rendering for unsupported policies.
Guest TLS documentation points at rejected driver tables	Fixed	Current-schema guidance keeps guest TLS fields under [openshell.gateway].
Kubernetes full example omits required callback configuration	Fixed	The raw TOML example now includes required grpc_endpoint and explains that Helm derives it from Service topology.
VM legacy-field rejection test can pass for the wrong reason	Fixed	Keep valid grpc_endpoint input while injecting openshell_endpoint, then assert the error names that unknown legacy field.
Broad prerelease release-script changes appear in the branch history	No action	Those changes come from mainline commit c8f13205; this PR's release-related edits are limited to schema-v2 packaging/config snippets.

Verification completed with targeted driver/server tests, Helm tests, mise run pre-commit, mise run test, and mise run ci. Podman E2E passed all applicable suites; only the environment-specific keep-id case was blocked because UID 16446 is not present in the host passwd database. Docker and VM runtime E2E were unavailable because this host has no Docker daemon or /dev/kvm.

[jhjaggars]

Schema-v2 follow-up completed

I implemented the remaining review findings while preserving the strict schema-v2 boundary.

Area	Resolution
Current main	Merged current main, resolved the openshell-gateway crate move, and retained the new factory/config ownership boundaries.
Homebrew upgrades	Formula startup rewrites only the exact empty or IPv6 schema-v1 config generated by prior formulas. Edited prefix and user configs remain untouched.
RPM upgrades	The user service now invokes a packaged migrator before startup. It seeds missing config, replaces only the byte-for-byte historical v1 default, rejects non-regular destinations, and preserves edited files.
Podman AppArmor	Omission now sends no override and preserves Podman's runtime-selected profile. Docker retains its explicit Unconfined default. The Podman development script opts into Unconfined explicitly.
Development pull policies	Docker, Podman, and generic gateway scripts normalize supported legacy/case variants such as IfNotPresent at the script boundary; schema-v2 TOML remains strict.
Guest TLS	Complete gateway-owned guest bundles are validated and injected before constructing local Docker, Podman, or VM drivers. TLS-enabled local drivers fail early without a bundle. Kubernetes continues to project TLS through its Secret and Helm does not render host guest-key paths.
JWT lifetime	SandboxJwtIssuer now carries Option<Duration> directly. None is non-expiring, Some(Duration::ZERO) is rejected, and extension credentials always use a finite bounded TTL.
VM gateway identity	[openshell.drivers.vm].sandbox_uid and sandbox_gid validate at the gateway and are forwarded to the VM subprocess.
VM persisted identity	Fresh identity markers are atomically installed before overlay creation. Restore prefers a validated marker, overlay upper-layer passwd, persisted prepared rootfs, explicit config, then the current image. Missing evidence fails safely; the driver never guesses 10001:10001, and ownership failures no longer fall back silently.
Documentation/RFC	Updated package docs, public references, architecture, driver READMEs, debugging/release skills, and RFC 0003 to match implemented schema-v2 behavior.

Verification

• mise run pre-commit
• mise run test
• mise run ci
• mise run helm:test — 131 tests passed
• mise run test:packaging-assets
• Homebrew/RPM migration pytest suite — 8 tests passed
• cargo test -p openshell-driver-podman — 221 tests passed
• cargo test -p openshell-driver-vm — 173 tests passed
• Focused gateway TLS/VM, server TLS/JWT/auth, Docker AppArmor, pull-policy, and shell syntax checks

The branch is now synchronized and pushed at a4acfb1c.