Skip to content

fix(driver-mxc): resolve StopSandbox by sandbox_id - #3278

Draft
letv1nnn wants to merge 2 commits into
NVIDIA:mainfrom
letv1nnn:3253-mxc-stop-sandbox-id/letv1nnn
Draft

fix(driver-mxc): resolve StopSandbox by sandbox_id#3278
letv1nnn wants to merge 2 commits into
NVIDIA:mainfrom
letv1nnn:3253-mxc-stop-sandbox-id/letv1nnn

Conversation

@letv1nnn

Copy link
Copy Markdown
Contributor

Summary

StopSandbox on the MXC driver discarded the sandbox_id the gateway sends and
resolved the target by name alone, over a registry that is shared across
workspaces. Because sandbox names are unique per workspace rather than globally,
a stop request could terminate a same-named sandbox in a different workspace,
chosen by hash iteration order. This makes sandbox_id authoritative for stop,
matching the contract DeleteSandbox already enforces on the same driver.

Related Issue

Fixes #3253

Changes

  • grpc.rs: StopSandbox now rejects an empty sandbox_id with InvalidArgument and forwards both identifiers to the backend. This mirrors the existing DeleteSandbox validation and removes name-only resolution rather than making it deterministic, matching the Podman and Kubernetes drivers. The gateway already populates sandbox_id on every stop call site, so no supported caller is affected.
  • driver.rs: MxcComputeBackend::stop_sandbox takes sandbox_id and resolves through registry.get(sandbox_id) instead of scanning values() for a name match, then returns FailedPrecondition when the resolved entry's name disagrees. This is the same id-first lookup plus name guard that delete_sandbox already uses. It also turns a linear scan into a hash lookup on the key the registry is already keyed by.
  • Tests: added a driver_sandbox_named helper so a sandbox's id, name, and workspace can be set independently. The existing helper forced name == id.to_string(), which made the whole class of id/name confusion bugs inexpressible in the suite. Added sandbox_by_id and wait_for_id because get_sandbox and wait_for resolve by name and cannot address an individual entry once two sandboxes share one.

Audited DeleteSandbox on this driver for the same weakness, per the issue's final acceptance criterion. It is already sound: id-first lookup with a FailedPrecondition name guard, and every subsequent operation keys on sandbox_id. No change needed. GetSandbox resolves by name but fails closed on an id mismatch, so it can return a wrong-workspace error but never mutates the wrong sandbox. StopSandbox was the only handler with no id gate at all.

Testing

  • mise run pre-commit passes
  • Unit tests added/updated
  • E2E tests added/updated (if applicable)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

…rd checkup

Signed-off-by: Artem Lytvyn <alytvyn@redhat.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 11, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@letv1nnn
letv1nnn marked this pull request as draft September 11, 2026 11:31

@sylvesterkaczmarek sylvesterkaczmarek left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The id/name validation happens before waiting on lifecycle_gate, but after the await the code only does registry.get_mut(sandbox_id) and does not verify it is still the entry whose gate/name was checked. A concurrent delete followed by recreation of the same id can therefore make this stop act on the replacement while holding the old entry’s gate. Could we revalidate the entry after acquiring the gate (including gate identity/name), and cover delete/recreate racing with stop?

…cle gate

Signed-off-by: Artem Lytvyn <alytvyn@redhat.com>

@sylvesterkaczmarek sylvesterkaczmarek left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked 1d4a5734. After waiting on the lifecycle gate, stop now revalidates that the registry entry still owns the same gate allocation before acting, and the delete/recreate regression covers reuse of the same sandbox id. This resolves the race I raised.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(driver-mxc): StopSandbox discards sandbox_id and can stop a same-named sandbox in another workspace

2 participants