feat: Add BoxLite VM runtime backend for hardware-level sandbox isolation

[DorianZheng]

Summary

Add BoxLite as an optional VM-based runtime backend for OpenShell sandboxes, providing hardware-level isolation alongside the existing Linux kernel isolation (Landlock + seccomp + netns).

Motivation

OpenShell currently relies on Linux kernel primitives for sandbox isolation:

• Landlock for filesystem access control
• seccomp for syscall filtering
• Network namespaces for network isolation

This has two limitations:

1. No macOS support: On non-Linux platforms, sandbox::apply() is a no-op that logs a warning. Developers on macOS have no isolation.
2. Kernel-level isolation only: For high-security workloads (regulated environments, multi-tenant), hardware-level VM isolation provides a stronger boundary — the guest runs its own kernel and cannot attack the host kernel.

BoxLite ("SQLite for sandboxing") fills both gaps:

• Runs on macOS ARM64 (Hypervisor.framework) and Linux (KVM)
• Provides hardware-level memory isolation via lightweight VMs
• Runs OCI container images inside VMs (same images OpenShell already uses)
• Async-first Rust API — same tech stack as OpenShell

Proposed Design

Runtime Backend Abstraction

Add a runtime module to openshell-sandbox with enum-based dispatch between backends:

openshell-sandbox/src/runtime/
├── mod.rs                 # RuntimeKind enum, SandboxedProcess enum, SpawnConfig
├── process_backend.rs     # Wraps existing ProcessHandle (default)
└── boxlite_backend.rs     # BoxLite VM backend (feature-gated)

Key types:

• RuntimeKind — enum (Process | Boxlite) selected via CLI --runtime flag or OPENSHELL_RUNTIME env var
• SandboxedProcess — enum wrapping either ProcessHandle or BoxliteProcess, with unified id(), wait(), signal(), kill() methods
• SpawnConfig — backend-agnostic spawn configuration

Integration Points

1. run_sandbox() gains a runtime_kind parameter
2. When RuntimeKind::Boxlite:
   • Skip Landlock/seccomp/netns setup (VM provides isolation)
   • Initialize BoxLite runtime
   • Create VM from the sandbox's container image
   • Execute agent command inside VM
3. Proxy/OPA/SSH/policy-poll continue working alongside either backend
4. Feature-gated behind boxlite Cargo feature (not compiled by default)

CLI Changes

openshell-sandbox --runtime boxlite [command...]
# or
OPENSHELL_RUNTIME=boxlite openshell-sandbox [command...]

What Changes, What Doesn't

Component	Process Backend	BoxLite Backend
Filesystem isolation	Landlock	VM boundary
Syscall filtering	seccomp	VM boundary (own kernel)
Network isolation	netns + veth	VM boundary
Proxy / OPA	✅ same	✅ same
SSH server	✅ same	✅ same
Policy engine	✅ same	✅ same
macOS support	❌ no-op	✅ full isolation

Dependencies

• boxlite crate (0.7.5, git dependency from github.com/boxlite-ai/boxlite)
• Feature-gated: [features] boxlite = ["dep:boxlite"]

Open Questions

1. Should BoxLite VM resources (CPU, memory) be configurable via sandbox spec or use sensible defaults?
2. Should the server-side (openshell-server) also be aware of the runtime kind, or is sandbox-level selection sufficient?
3. How should port forwarding from the proxy into the BoxLite VM be handled?