test: add a hosted-runner trezor-emu path alongside the self-hosted one

[mdozhdev]

Description

trezor-emu needs Docker and the Simulator on one machine, so it runs on the self-hosted Mac — the last job keeping it alive.

Adds the same suite as a second suite choice: emulator stack on ubuntu-latest, Simulator on macos-latest, joined over Tailscale. Same shape as #689.

The suite reaches Bridge, the User Env controller, electrs and bitcoind as 127.0.0.1 from inside the Simulator, so the Mac job relays those ports onto loopback rather than making the suite stack-aware. No Swift or Info.plist changes.

trezor-emu is untouched and still dispatchable. Dispatch-only, so nothing runs unless asked.

Linked Issues/Tasks

• synonymdev/pubky-stack#275

[greptile-apps]

Greptile Summary

The PR adds a dispatch-only hosted Trezor emulator path that runs the Docker stack on Ubuntu and the iOS Simulator tests on macOS, connecting them through Tailscale.

• Adds independent Linux stack and macOS test jobs with coordinated lifecycle polling.
• Funds a fresh regtest wallet and relays stack services onto the Mac runner's loopback interface.
• Collects diagnostics from both runners.

Confidence Score: 3/5

The PR should not merge until the external stack revision is pinned because the new job executes mutable upstream code after joining the project tailnet.

The hosted stack path introduces a concrete supply-chain execution boundary, while its readiness checks also make unused ports a non-blocking source of workflow failure.

Files Needing Attention: .github/workflows/ai-device-tests.yml

Security Review

The Linux job executes scripts from the mutable bitkit-docker main branch after becoming a tailnet member, allowing an upstream compromise to execute in a credentialed and network-connected CI context.

Important Files Changed

Filename	Overview
.github/workflows/ai-device-tests.yml	Adds the complete two-runner Trezor test path; the mutable external checkout creates a supply-chain boundary issue, and the readiness gate unnecessarily couples the suite to two unused ports.

Sequence Diagram

sequenceDiagram
  participant L as Ubuntu stack job
  participant T as Tailscale
  participant M as macOS test job
  participant S as iOS Simulator
  L->>T: Join as trezor-stack-run_id
  L->>L: Start and fund emulator stack
  M->>T: Join as trezor-tester-run_id
  M->>T: Discover stack runner
  M->>L: Probe stack ports
  M->>M: Relay ports onto 127.0.0.1
  S->>M: Connect to loopback services
  M->>S: Run Trezor UI tests
  L->>L: Poll until test job completes
  L->>L: Collect diagnostics and stop stack

Loading

_{Reviews (1): Last reviewed commit: "Merge branch 'master' into test/trezor-e..." | Re-trigger Greptile}

[greptile-apps]

Mutable code runs inside tailnet

When bitkit-docker's main branch changes or is compromised before dispatch, this job executes that unreviewed revision after joining the project tailnet, exposing CI resources, reachable tailnet services, and the checkout-persisted GitHub token to arbitrary upstream code. Pin the checkout to a reviewed commit SHA. How this was verified: The workflow checks out mutable main, joins Tailscale with TS_AUTHKEY, and then runs docker compose and ./scripts/trezor-emulator from that checkout.

[jvsena42]

🔵 TS_AUTHKEY now lands on GitHub-hosted runners — confirm it is ephemeral, tagged and ACL-scoped

The self-hosted trezor-emu path consumes no secrets at all. Both new jobs put a Tailscale auth key on shared GitHub-hosted runners, and anything that can inject a step into this workflow (or any third-party action running in these jobs) can exfiltrate it and join your tailnet. Confirm TS_AUTHKEY is reusable-but-ephemeral, pre-authorized, and carries a dedicated tag whose ACL grants only the regtest/Trezor stack ports — not a general-purpose key. If it is not ephemeral, every run also leaves a dead node behind, which breaks the HostName == peer lookup on the next run.

[greptile-apps]

I can’t confirm those properties from this PR. The workflow only references ${{ secrets.TS_AUTHKEY }}; GitHub does not expose whether that key is reusable, ephemeral, pre-authorized, or tagged, and the repository contains no Tailscale ACL configuration.

The required configuration should be reusable (so both jobs can enroll), ephemeral (so each runner node is removed after logout/inactivity), pre-authorized (if device approval is enabled), and assigned a dedicated tag whose grants permit only the required regtest/Trezor endpoints. The ACL should not grant general tailnet access. A reusable but non-ephemeral key would leave nodes behind and make the HostName == peer lookup unreliable.

Please verify the key settings and tag/grant policy in Tailscale administration, or use per-run OIDC/OAuth-generated ephemeral credentials. Until that is confirmed, this security concern remains unresolved.

[greptile-apps]

Unused ports gate test readiness

The readiness loop makes ports 9002 and 6080 mandatory even though the current suite uses only 21325, 9001, 43782, and 60001. Losing either unused endpoint therefore delays the job for up to 30 minutes and fails an otherwise healthy test stack; remove them from both the readiness probe and relay list.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[jvsena42]

workflow_dispatch only, no pull_request_target, no untrusted checkout, and the concurrency group already includes inputs.suite, so the two paths cannot collide on the emulator.

One medium: the hold loop can fall out of its deadline and report success. Note greptile raised this exact defect on #689, but quoting this PR's 150/180 numbers — it applies here, not there. #689's sibling job has the guard this one is missing.

[jvsena42]

🟡 Hold loop tears the stack down mid-test and reports success

The loop exits on deadline with no guard, so when it expires while trezor-emu-remote is still running the next steps stop the emulator and docker compose down, and this job goes green. The tester then fails with connection errors and the stack job looks healthy. #689's regtest-stack has exactly this guard (finished flag + exit 1); add it here too. Also raise the hold deadline: 9000s is under the 180-minute timeout-minutes on trezor-emu-remote, so a run that uses its full budget is guaranteed to hit this.

[jvsena42]

🔵 Add permissions: — the jobs API needs actions: read, and the error is swallowed

2>/dev/null || echo 1 turns any gh api failure into 'tests still running', so a broken API call is indistinguishable from a slow suite and the runner idles to its deadline with no signal. Let the failure surface, or at least echo "::warning::jobs API failed" on the fallback. While here, add permissions: { contents: read, actions: read } to this job so it does not depend on the repo-wide default token scope.

[jvsena42]

🔵 Pin tailscale/github-action to a commit SHA

@v3 is a mutable tag on a third-party repo, and this is the one action in the workflow that is handed a credential and rewrites the runner's network stack. Pin it to a full commit SHA (tailscale/github-action@<sha> # v3.x.y). Applies to both occurrences (lines 154 and 296).