Skip to content

Qualify Rust namespace execution and role credentials - #159

Merged
rmcdaniel merged 9 commits into
mainfrom
test/rust-namespace-isolation
Oct 8, 2026
Merged

rmcdaniel merged 9 commits into
mainfrom
test/rust-namespace-isolation

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

Purpose

Part of durable-workflow/.github#122. Exercise namespace isolation through the published Rust SDK's real clients and workers.

Changes

  • Reuse the existing Rust worker image and public Server stack for two namespaces with identical queue and type names. Verify the Server's explicit cross-namespace workflow-ID reservation with HTTP 409, and use distinct IDs for successful runs.
  • Use namespace-bound operator-only callers and worker-only workers. A dual-credential client checks request-plane credential selection.
  • Check typed missing-role errors and exact Server 403/404 reasons for wrong-role and cross-namespace reads, signals, starts, registration and polls.
  • Execute four Rust workflows and remote activities. SIGKILL both workers, acknowledge signals while they are absent, and replace them with fresh containers.
  • Verify original run, activity execution/attempt and signal-wait identities, namespace-specific results and exactly one completion.
  • Add scripts/sdk-namespaces.sh, a thin hosted workflow, focused history validation checks and a public runnable guide. Cleanup runs on success and failure.
  • Add optional bounded PHP worker diagnostics and a read-only timer recovery snapshot. Adopt published PHP SDK 2.2.4, whose heartbeat recovery fix was discovered by the existing timer gate during this work.

Qualification

Local Rust 1.86 compilation and all five history-validation tests pass. The final candidate 3e5178dcc24ee190b25e64f7cbd07dde031537ce and merged main add1cab85b7854fcc2eac719fb7df98f9ee82d18 pass every gate. Namespace execution records 52 observations, including four original runs recovered and 28 negative credential/namespace operations. All twelve timer scenarios pass with published PHP 2.2.4.

The frozen tuple uses Server 2.5.9, PHP SDK 2.2.4, Rust SDK 3.3.3, Python SDK 2.4.3, Workflow 2.5.3, Waterline 2.3.1 and CLI 2.2.0. No SDK or Server source overlay is used. These are disposable fixture credentials and namespaces. Namespace administration remains the public Server API.

PHP SDK recovery fix: durable-workflow/sdk-php#105

The tuple and all three PHP consumer lock files select 2.2.4. No other locked package version changes. Both prepared image architectures were rebuilt and passed anonymous pull/startup qualification before main promotion. Direct anonymous GHCR and Docker Hub reads verify matching immutable/main index sha256:3e3c4127d57385c4a2261740be56851271d261d76f53fbb5ded3d2a86d2e9200 and image source labels matching the merged commit.

Published qualification: https://github.com/durable-workflow/sample-app/actions/runs/37737665828

@rmcdaniel
rmcdaniel marked this pull request as ready for review October 8, 2026 05:34
@rmcdaniel

Copy link
Copy Markdown
Member Author

The published Rust namespace experiment passed on the candidate, including four original runs recovered after both workers were killed and recreated, separate operator and worker credentials, and 28 rejected credential or namespace operations.

Verified run: https://github.com/durable-workflow/sample-app/actions/runs/37733683629

Merge is held because the existing timer gate found that the published PHP worker stopped recovering after Server restarted. Python and Rust recovered their original runs. The current candidate adds bounded worker diagnostics and a read-only registry/history snapshot to identify the PHP failure. It preserves the timer timeout and recovery procedure.

Failed timer run: https://github.com/durable-workflow/sample-app/actions/runs/37733683535

The current update job failed while fetching Cargo dependencies before execution. Its failed job is being retried. Namespace qualification on the diagnostic candidate has also passed.

@rmcdaniel

Copy link
Copy Markdown
Member Author

All gates now pass on namespace candidate c9089d1e675972e051e9d9e5693b10fdd190a652, including the twelve timer scenarios. The earlier PHP timer recovery failure remains real and recorded.

A deterministic PHP SDK test reproduced an outage ordering defect: task acquisition can resume while a failed registration heartbeat is still waiting for its retry deadline, then stop permanently on a stale registration poll response. The SDK fix is in durable-workflow/sdk-php#105 and the customer outcome is tracked in durable-workflow/sdk-php#104.

This qualification will adopt and verify the published patch before merge. The diagnostic run passing does not establish that the earlier defect was fixed.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Reviewed final source 3e5178dcc24ee190b25e64f7cbd07dde031537ce against main 2e8b651bf1b384f5b16b0cd3890f78d755ba20e7.

The experiment uses published packages and Server, isolated fixture credentials, two namespaces with the same queue/type names, and real Rust client and worker requests. It checks role selection and explicit refusals, executes remote activities, kills both workers, acknowledges signals while they are absent, and verifies original durable identities after fresh replacement. Cross-namespace workflow ID reservation follows the Server's documented refusal contract.

The observer rejects lost or duplicate durable boundaries, foreign results and replacement identities. Five observer tests cover these failures. Cleanup removes experiment containers, networks, volumes and fixture images on success and failure. The optional PHP diagnostic listener includes only bounded protocol fields, with no tokens, payloads or raw exception bodies.

Final published tuple: Server 2.5.9, PHP SDK 2.2.4, Python SDK 2.4.3, Rust SDK 3.3.3, Workflow 2.5.3, Waterline 2.3.1 and CLI 2.2.0. The three PHP consumer locks select the published 2.2.4 commit without changing other package versions.

Passing connected runs:

App, microservice, polyglot and smoke checks also passed. Candidate image startup is the remaining gate. After merge, rebuild and verify both published image architectures through the existing workflow, then publish the dependent coverage documentation.

@rmcdaniel
rmcdaniel merged commit add1cab into main Oct 8, 2026
15 checks passed
@rmcdaniel
rmcdaniel deleted the test/rust-namespace-isolation branch October 8, 2026 06:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant