Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 156 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,162 @@ go build -o oz-agent-worker
./oz-agent-worker --api-key "wk-abc123" --worker-id "my-worker"
```

## Security and Sandboxing

Oz agents run inside isolated containers on the worker host. The table below summarizes
what each backend isolates and what it does not:

| Isolation dimension | Docker backend | Kubernetes backend | Direct backend |
|---|---|---|---|
| Process | ✅ Container per task | ✅ Pod per task | ⚠️ Host process |
| Filesystem | ✅ Ephemeral container FS | ✅ Ephemeral pod FS | ⚠️ Host filesystem |
| Resource (CPU/memory) | ✅ Limits when runner shape set | ✅ Requests/limits when runner shape set | ❌ None |
| Network egress | ⚠️ Default bridge (unrestricted) | ⚠️ Cluster default (see NetworkPolicy) | ❌ None |

**The Direct backend provides no container boundary.** Tasks run as host processes with
access to the host network, filesystem, and process space. It is not appropriate for
environments where agents must be prevented from reaching internal infrastructure.

### Docker: Restricting network access

By default, each task container joins Docker's default bridge network, which gives the
container outbound connectivity to anything the worker host can reach — including internal
infrastructure. You can restrict this in two ways:

#### 1. Isolated Docker network (`network_mode`)

Set `backend.docker.network_mode` to attach task containers to a specific Docker network
instead of the default bridge:

```yaml
worker_id: "my-worker"
backend:
docker:
network_mode: "none" # no networking; agent cannot make outbound connections
```

Or attach to a restricted network you manage:

```yaml
backend:
docker:
network_mode: "restricted-net" # your Docker network with limited egress
```

Accepted values are any string accepted by Docker's `--network` flag:
- `"none"` — no networking at all (maximum isolation)
- A named network — attaches the container to that network
- `"host"` — full host network access (expands rather than restricts egress; not recommended for security-sensitive use)
- Empty string / omitted — Docker's default bridge (current behavior)

The worker passes the value to Docker verbatim; an invalid name surfaces as a task
failure at container creation time.

#### 2. Egress proxy (`HTTP_PROXY` / `HTTPS_PROXY`)

Route all HTTP/HTTPS traffic through a filtering proxy you operate by injecting the
standard proxy environment variables:

```yaml
backend:
docker:
environment:
- name: HTTP_PROXY
value: "http://proxy.internal.example.com:3128"
- name: HTTPS_PROXY
value: "http://proxy.internal.example.com:3128"
- name: NO_PROXY
value: "oz.warp.dev,app.warp.dev,localhost,127.0.0.1"
```

Include `oz.warp.dev` and `app.warp.dev` in `NO_PROXY` so the worker's WebSocket
connection and agent task-status calls bypass the proxy.

**Limitations:** `HTTP_PROXY`/`HTTPS_PROXY` are honored by most HTTP client libraries
(Go, Python, Node.js) but not by raw TCP connections, UDP, or DNS resolvers. For full
protocol coverage, combine the proxy with a restricted Docker network and host-level
iptables rules.

### Kubernetes: NetworkPolicy

Task pods are labeled with `oz-task-id` and `oz-worker-id` by the worker. You can write
a `NetworkPolicy` that selects on these labels to restrict what task pods can reach:

```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: restrict-oz-task-egress
namespace: agents
spec:
podSelector:
matchLabels:
oz-worker-id: my-worker
policyTypes:
- Egress
egress:
# Allow task pods to reach Warp control plane
- to:
- ipBlock:
cidr: 0.0.0.0/0
except:
- 10.0.0.0/8 # block RFC-1918 internal ranges
- 172.16.0.0/12
- 192.168.0.0/16
ports:
- protocol: TCP
port: 443
```

NetworkPolicy enforcement requires a CNI plugin that implements it (Calico, Cilium,
Weave, etc.). The default kubenet and flannel CNI plugins do not enforce NetworkPolicy
resources. The worker does not create NetworkPolicy resources; you manage them in your
cluster.

You can also use `backend.kubernetes.pod_template` to set pod-level security contexts,
service accounts with restricted permissions, or custom labels for policy selectors:

```yaml
backend:
kubernetes:
namespace: agents
pod_template:
metadata:
labels:
security-tier: restricted
containers:
- name: task
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
```

For Kubernetes, you can also inject proxy env vars via `pod_template`:

```yaml
backend:
kubernetes:
pod_template:
containers:
- name: task
env:
- name: HTTP_PROXY
value: "http://proxy.internal.example.com:3128"
- name: HTTPS_PROXY
value: "http://proxy.internal.example.com:3128"
- name: NO_PROXY
value: "oz.warp.dev,app.warp.dev,10.0.0.0/8"
```

### Warp-hosted cloud agents

Warp-hosted runs execute in ephemeral Namespace.so microVMs that are strongly isolated
from the host and from other tenants. These sandboxes cannot reach customer internal
infrastructure by default. Enterprise customers who need agents to access internal
services use self-hosted workers.

## Environment Variables for Task Containers

Use `-e` / `--env` to pass environment variables into task containers:
Expand Down
5 changes: 5 additions & 0 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ type BackendConfig struct {
type DockerConfig struct {
Volumes []string `yaml:"volumes"`
Environment []EnvEntry `yaml:"environment" validate:"dive"`
// NetworkMode sets the Docker network the task container joins.
// Accepts any value Docker's --network flag accepts: a named network
// (e.g. "restricted-net"), "none" (no networking), "host", or empty
// string (Docker's default bridge network).
NetworkMode string `yaml:"network_mode"`
}

// DirectConfig holds direct-backend-specific configuration.
Expand Down
16 changes: 11 additions & 5 deletions internal/worker/docker.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,14 @@ const dockerHubAuthConfigKey = "https://index.docker.io/v1/"

// DockerBackendConfig holds configuration specific to the Docker backend.
type DockerBackendConfig struct {
NoCleanup bool
Volumes []string
Env map[string]string
NoCleanup bool
Volumes []string
Env map[string]string
// NetworkMode sets the Docker network the task container joins.
// Accepts any value Docker's --network flag accepts: a named network
// (e.g. "restricted-net"), "none" (no networking), "host", or empty
// string (Docker's default bridge network).
NetworkMode string
}

func (b *DockerBackend) containerWasOOMKilled(ctx context.Context, dockerClient *client.Client, containerID string) bool {
Expand Down Expand Up @@ -137,8 +142,9 @@ func (b *DockerBackend) ExecuteTask(ctx context.Context, params *TaskParams) err
binds = append(binds, b.config.Volumes...)

hostConfig := &container.HostConfig{
Binds: binds,
Resources: dockerResourcesForShape(params.InstanceShape),
Binds: binds,
Resources: dockerResourcesForShape(params.InstanceShape),
NetworkMode: container.NetworkMode(b.config.NetworkMode),
}

resp, err := dockerClient.ContainerCreate(ctx, client.ContainerCreateOptions{
Expand Down
13 changes: 10 additions & 3 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -319,10 +319,17 @@ func mergeConfig(fileConfig *config.FileConfig) (worker.Config, error) {
}
volumes = append(volumes, CLI.Volumes...)

// Resolve network_mode: only from config file (no CLI flag).
var networkMode string
if fileConfig != nil && fileConfig.Backend.Docker != nil {
networkMode = fileConfig.Backend.Docker.NetworkMode
}

wc.Docker = &worker.DockerBackendConfig{
NoCleanup: noCleanup,
Volumes: volumes,
Env: mergedEnv,
NoCleanup: noCleanup,
Volumes: volumes,
Env: mergedEnv,
NetworkMode: networkMode,
}
}

Expand Down
167 changes: 167 additions & 0 deletions specs/security-sandboxing/PRODUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
# Security and Sandboxing for Autonomous Oz Agents

## Summary

Enterprise customers running self-hosted Oz workers need clarity on the security and
sandboxing guarantees Oz provides out-of-the-box, and reliable mechanisms to restrict
what autonomous agents can reach on their internal infrastructure. Today the Docker and
Kubernetes backends provide process/filesystem isolation but expose no native controls
for network egress or filtering. This spec documents the current isolation boundary,
identifies enterprise gaps, and introduces two practical additions: a configurable Docker
network mode so task containers can be attached to a restricted or isolated network, and
first-class documentation of egress proxy configuration for both backends.

## Background: What Oz Provides Today

### Warp-hosted cloud agents (Namespace.so sandboxes)

Each cloud agent run executes inside an ephemeral Namespace.so microVM. The sandbox
provides strong OS-level isolation: the agent cannot escape the VM boundary, cannot
address other tenant workloads, and cannot reach customer internal infrastructure unless
the customer explicitly exposes an endpoint. This model offers robust out-of-the-box
isolation but provides no mechanism for the agent to reach internal infrastructure at all,
which is why enterprise customers need self-hosted workers.

### Self-hosted workers — Docker backend

- **Process isolation:** Each task runs inside a distinct Docker container (one container
per task). The container lifecycle is tied to the task; containers are removed after
completion by default.
- **Filesystem isolation:** The container has its own ephemeral filesystem. Only volumes
explicitly mounted by the operator are accessible.
- **Resource isolation:** CPU and memory limits are applied when a runner instance shape
is configured (REMOTE-1936 / merged). Without a shape the container runs unconstrained.
- **Network isolation (current gap):** By default the container is attached to Docker's
default bridge network. This gives the agent unrestricted outbound connectivity from the
worker host, including access to internal infrastructure reachable from that host. No
egress filtering is applied by the worker; no allowlist or denylist of destinations is
enforced.

### Self-hosted workers — Kubernetes backend

- **Process isolation:** Each task runs as a Kubernetes Job (one pod per task). Pod and
container boundaries provide OS-level isolation.
- **Filesystem isolation:** Ephemeral `emptyDir` volumes; operator-mounted volumes only.
- **Namespace isolation:** The worker deploys task pods in a configured namespace. The
operator can separate task pods from other workloads via Kubernetes RBAC and namespace
boundaries.
- **Network isolation (operator-controlled):** Kubernetes NetworkPolicy resources can be
applied at the namespace level to restrict what task pods can reach. The worker's
`pod_template` field already accepts arbitrary PodSpec YAML, so operators can add pod
labels that a NetworkPolicy selector can match. NetworkPolicy enforcement depends on the
cluster's CNI plugin (Calico, Cilium, etc.). No NetworkPolicy is created by the worker
itself; the operator configures this entirely in their cluster.
- **Security contexts:** The Helm chart applies hardened defaults to the long-lived worker
Deployment pod: `runAsNonRoot`, `seccompProfile: RuntimeDefault`, dropped capabilities.
Task pods inherit defaults from the cluster unless the operator's `pod_template`
overrides them.

### Self-hosted workers — Direct backend

- **No container boundary:** Tasks run as a subprocess of the worker process on the host.
All host network access is available. No sandboxing is provided beyond process
isolation.
- **Direct backend is not appropriate for restricted-access scenarios** without the
operator layering their own OS-level controls (firewall rules, seccomp, etc.).

## Problem

An IMC enterprise evaluator asked whether Oz provides out-of-the-box network-level
security controls — network policies, filtering proxies — so autonomous agents cannot
freely reach internal infrastructure. The honest answer today is: we provide strong
process/filesystem isolation but not network-level egress filtering. Enterprise customers
must implement their own controls (Kubernetes NetworkPolicy, egress firewall rules, or a
filtering proxy). They need:

1. **Clarity** on what is and is not provided today, so they can design their controls
correctly.
2. **A supported, documented egress proxy pattern** so agents can route all outbound
traffic through a filtering proxy the customer operates.
3. **Docker network isolation** — an explicit hook to attach task containers to a
restricted Docker network (instead of the default bridge) without requiring the
operator to patch entrypoints or iptables rules manually.

## Goals

- Document the current isolation boundary clearly in the README and the
self-hosted-worker customer guidance skill.
- Add a `network_mode` field to the Docker backend configuration so operators can specify
the Docker network name (or `none`) that task containers join. This is the practical
hook for connecting containers to a restricted network managed by the operator.
- Document the egress proxy pattern: passing `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY`
to task containers via the existing environment variable injection mechanism.
- Document the Kubernetes NetworkPolicy pattern using the existing `pod_template` hook.

## Non-goals

- Warp does not implement a built-in filtering proxy, firewall ruleset, or content
inspection layer. These belong to customer infrastructure.
- Warp-hosted sandbox network restrictions are out of scope; customers who need to reach
internal infra already use self-hosted workers.
- No Oz server changes are required; all changes are in `oz-agent-worker`.
- The Direct backend is out of scope for network isolation; it has no container boundary
the worker can configure.

## Behavior

### Docker network mode

1. The Docker backend config accepts an optional `network_mode` string. When set, the
task container's `HostConfig.NetworkMode` is set to that value.
2. Accepted values are any value Docker's `--network` flag accepts: a named network
(e.g. `restricted-net`, `host`, `none`), or the empty string (default bridge).
3. When `network_mode` is absent or empty, behavior is identical to today (Docker's
default bridge network).
4. Setting `network_mode: none` disables all networking for the task container. The
agent runtime can still reach the Warp control plane through the worker's own network
path; task-internal tool calls that require internet access will fail, which is the
intended behavior for maximum isolation.
5. Operators who want egress-filtered access (rather than no access) create a Docker
network with appropriate iptables or firewall rules and set `network_mode` to that
network's name. The worker does not manage the network itself.

### Egress proxy

6. Task containers respect the standard `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY`
environment variables. Operators inject these via the existing `backend.docker.environment`
config entries or Kubernetes `pod_template` `env` entries.
7. A filtering proxy the operator operates receives all outbound HTTP/HTTPS traffic from
the agent and can enforce an allowlist or denylist of destinations.
8. `NO_PROXY` should include the Warp control plane endpoints (`oz.warp.dev`,
`app.warp.dev`) so the worker's own WebSocket and the agent's task-status calls do
not route through the proxy.
9. The worker does not validate or enforce proxy settings; responsibility lies with the
operator. This is documented explicitly so customers do not assume the worker
intercepts non-proxy-aware traffic.

### Kubernetes NetworkPolicy

10. Task pods are labeled with `oz-task-id` and `oz-worker-id` by the worker. Operators
can write NetworkPolicy resources that select on these labels to restrict task pod
egress to a set of allowed destinations.
11. No NetworkPolicy is created by the worker. Operators are responsible for creating and
maintaining NetworkPolicy resources that match their trust requirements.
12. The `pod_template` mechanism already supports adding arbitrary pod metadata (labels,
annotations) that NetworkPolicy selectors and admission controllers can consume.
Operators can also add pod-level security settings (securityContext, seccompProfile,
capabilities) via `pod_template`.

### Documentation

13. The README security section documents:
- The isolation model per backend (Docker, Kubernetes, Direct).
- The `network_mode` option for Docker.
- The egress proxy pattern.
- The Kubernetes NetworkPolicy guidance.
- An explicit statement that the Direct backend provides no network isolation.
14. The self-hosted-worker customer guidance skill's `secrets-and-security.md` reference
is updated to match.

## Why not build a built-in egress filter?

A built-in filtering layer in the worker (e.g. a sidecar transparent proxy enforced by
iptables) would require elevated capabilities (`NET_ADMIN`), is hard to keep correct
across Linux kernel versions, and duplicates tools that enterprise operators already run
(Zscaler, Squid, Cilium Egress Gateway). The proxy pattern and Kubernetes NetworkPolicy
are lower-risk, more auditable, and compose with existing enterprise controls. We document
how to use them with Oz rather than reimplementing them.
Loading
Loading