test: add a hosted-runner trezor-emu path alongside the self-hosted one #692
+363
−0
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,6 +10,7 @@ on: | |
| type: choice | ||
| options: | ||
| - trezor-emu | ||
| - trezor-emu-remote | ||
| simulator_name: | ||
| description: "iOS Simulator name" | ||
| required: false | ||
|
|
@@ -134,3 +135,356 @@ jobs: | |
| run: | | ||
| ./scripts/trezor-emulator stop || true | ||
| docker compose down || true | ||
|
|
||
| # The emulator stack for trezor-emu-remote, on a runner that can run Docker. | ||
| # Must not depend on trezor-emu-remote, nor it on this: this job only finishes | ||
| # once the tests are done, so a dependency either way deadlocks. | ||
| trezor-stack: | ||
| if: inputs.suite == 'trezor-emu-remote' | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 180 | ||
|
|
||
| steps: | ||
| - name: Checkout bitkit-docker | ||
| uses: actions/checkout@v7 | ||
| with: | ||
| repository: synonymdev/bitkit-docker | ||
| ref: main | ||
|
|
||
| - uses: tailscale/github-action@v3 | ||
|
Member
There was a problem hiding this comment. 🔵 Pin tailscale/github-action to a commit SHA
|
||
| with: | ||
| authkey: ${{ secrets.TS_AUTHKEY }} | ||
| hostname: trezor-stack-${{ github.run_id }} | ||
| args: --accept-dns=false | ||
|
|
||
| - name: Start regtest and Trezor emulator | ||
| run: | | ||
| set -euo pipefail | ||
| echo "tailnet address: $(tailscale ip -4)" | ||
|
|
||
| # Naming the services: trezor-user-env-mac carries no `profiles` key, | ||
| # so a bare `up -d` starts it too and it takes 21325 before the | ||
| # host-networked Linux service the helper selects here can bind it. | ||
| docker compose up -d bitcoind bitcoinsetup electrs darkhttpd | ||
| ./scripts/trezor-emulator start | ||
| docker compose ps | ||
|
|
||
| wait_for() { | ||
| local what=$1 deadline=$(( SECONDS + 300 )) | ||
| until eval "$2"; do | ||
| if (( SECONDS >= deadline )); then | ||
| echo "::error::timed out waiting for $what" | ||
| docker compose logs --no-color --tail=50 | ||
| exit 1 | ||
| fi | ||
| sleep 5 | ||
| done | ||
| echo "✓ $what" | ||
| } | ||
|
|
||
| wait_for "electrs on 60001" 'nc -z 127.0.0.1 60001' | ||
| wait_for "bitcoind rpc on 43782" 'nc -z 127.0.0.1 43782' | ||
| # Bridge answers on 21325 before it has the emulator, so wait on the | ||
| # device rather than on the port. | ||
| wait_for "a trezor device on the bridge" \ | ||
| 'curl -fsS -m 10 -X POST http://127.0.0.1:21325/enumerate | grep -q "\"path\""' | ||
|
|
||
| # A coinbase matures after 100 confirmations and the compose setup mines a | ||
| # single block, so nothing is spendable and the suite cannot fund a Trezor | ||
| # address. On a developer machine the chain volume carries blocks over | ||
| # between runs, which hides this; a runner always starts at height 1. | ||
| - name: Fund the regtest wallet | ||
| run: | | ||
| set -euo pipefail | ||
| rpc() { | ||
| curl -fsS --user polaruser:polarpass -H 'content-type: application/json' \ | ||
| --data "{\"jsonrpc\":\"1.0\",\"id\":\"ci\",\"method\":\"$1\",\"params\":$2}" \ | ||
| http://127.0.0.1:43782/ | ||
| } | ||
|
|
||
| address=$(rpc getnewaddress '[]' | jq -r '.result') | ||
| rpc generatetoaddress "[101, \"$address\"]" > /dev/null | ||
|
|
||
| balance=$(rpc getbalance '[]' | jq -r '.result') | ||
| echo "spendable balance: $balance" | ||
| echo "$balance" | jq -e '. > 0' > /dev/null | ||
|
|
||
| - name: Hold the stack up until the tests finish | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
|
Member
There was a problem hiding this comment. 🔵 Add
|
||
| run: | | ||
| set -euo pipefail | ||
| deadline=$(( SECONDS + 9000 )) | ||
| while (( SECONDS < deadline )); do | ||
| pending=$(gh api "repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/jobs" \ | ||
| --paginate --jq '[.jobs[] | select(.name == "trezor-emu-remote") | select(.status != "completed")] | length' \ | ||
| 2>/dev/null || echo 1) | ||
| started=$(gh api "repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/jobs" \ | ||
| --paginate --jq '[.jobs[] | select(.name == "trezor-emu-remote")] | length' \ | ||
| 2>/dev/null || echo 0) | ||
| echo "trezor-emu-remote: ${started} job(s), ${pending} still running" | ||
| if [ "$started" -gt 0 ] && [ "$pending" -eq 0 ]; then | ||
| echo "tests finished" | ||
| break | ||
| fi | ||
| sleep 30 | ||
| done | ||
|
Member
There was a problem hiding this comment. 🟡 Hold loop tears the stack down mid-test and reports success The loop exits on |
||
|
|
||
| - name: Collect stack diagnostics | ||
| if: always() | ||
| run: | | ||
| mkdir -p ai-device-artifacts/trezor-stack | ||
| ./scripts/trezor-emulator status > ai-device-artifacts/trezor-stack/trezor-status.json 2>&1 || true | ||
| curl --silent --show-error -X POST http://127.0.0.1:21325/enumerate > ai-device-artifacts/trezor-stack/bridge-enumerate.json 2>&1 || true | ||
| docker compose --profile trezor-linux logs --no-color --tail=500 trezor-user-env-linux > ai-device-artifacts/trezor-stack/trezor-user-env.log 2>&1 || true | ||
| docker compose logs --no-color > ai-device-artifacts/trezor-stack/docker-compose.log 2>&1 || true | ||
|
|
||
| - name: Upload stack diagnostics | ||
| if: always() | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: ai-device-tests-trezor-stack-${{ github.run_number }} | ||
| path: ai-device-artifacts/trezor-stack | ||
| if-no-files-found: warn | ||
|
|
||
| - name: Stop emulator services | ||
| if: always() | ||
| run: | | ||
| ./scripts/trezor-emulator stop || true | ||
| docker compose down || true | ||
|
|
||
| # Same suite as trezor-emu, on a GitHub-hosted Mac with the emulator stack on | ||
| # another runner. Runs alongside it until it has earned replacing it. | ||
| trezor-emu-remote: | ||
| if: inputs.suite == 'trezor-emu-remote' | ||
| runs-on: macos-latest | ||
| timeout-minutes: 180 | ||
|
|
||
| steps: | ||
| - name: Checkout Bitkit iOS | ||
| uses: actions/checkout@v7 | ||
|
|
||
| - name: Set up Xcode | ||
| uses: maxim-lobanov/setup-xcode@v1 | ||
| with: | ||
| xcode-version: "26.2" | ||
|
|
||
| - name: Install xcbeautify | ||
| run: | | ||
| if ! command -v xcbeautify >/dev/null 2>&1; then | ||
| brew install xcbeautify | ||
| fi | ||
|
|
||
| - name: System information | ||
| run: | | ||
| sw_vers | ||
| xcodebuild -version | ||
|
|
||
| - name: Cache Swift Package Manager | ||
| uses: actions/cache@v6 | ||
| with: | ||
| path: | | ||
| ~/Library/Caches/org.swift.swiftpm | ||
| ~/Library/org.swift.swiftpm | ||
| Bitkit.xcodeproj/project.xcworkspace/xcshareddata/swiftpm | ||
| key: ${{ runner.os }}-spm-${{ hashFiles('**/Package.resolved') }} | ||
|
|
||
| - name: Resolve Swift packages | ||
| run: | | ||
| xcodebuild -resolvePackageDependencies -onlyUsePackageVersionsFromResolvedFile | xcbeautify | ||
|
|
||
| - uses: tailscale/github-action@v3 | ||
| with: | ||
| authkey: ${{ secrets.TS_AUTHKEY }} | ||
| hostname: trezor-tester-${{ github.run_id }} | ||
| # Peers are found via `tailscale status`, so MagicDNS is unused. Leaving | ||
| # it on rewrites the resolver, and its macOS setup targets a network | ||
| # service named "Ethernet" that hosted runners do not have. | ||
| args: --accept-dns=false | ||
|
|
||
| - name: Find the stack runner | ||
| run: | | ||
| set -euo pipefail | ||
| deadline=$(( SECONDS + 1800 )) | ||
| while :; do | ||
| ip=$(tailscale status --json 2>/dev/null \ | ||
| | jq -r --arg h "trezor-stack-${{ github.run_id }}" \ | ||
| 'first(.Peer[]? | select(.HostName == $h) | .TailscaleIPs[0]) // empty' \ | ||
| || true) | ||
| [ -n "$ip" ] && break | ||
| if (( SECONDS >= deadline )); then | ||
| echo "::error::stack runner never joined the tailnet" | ||
| tailscale status || true | ||
| exit 1 | ||
| fi | ||
| sleep 10 | ||
| done | ||
| echo "STACK_IP=$ip" >> "$GITHUB_ENV" | ||
|
|
||
| for port in 21325 9001 43782 60001 9002 6080; do | ||
|
There was a problem hiding this comment.
The readiness loop makes ports 9002 and 6080 mandatory even though the current suite uses only 21325, 9001, 43782, and 60001. Losing either unused endpoint therefore delays the job for up to 30 minutes and fails an otherwise healthy test stack; remove them from both the readiness probe and relay list. Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! |
||
| until nc -z -w 5 "$ip" "$port" 2>/dev/null; do | ||
| if (( SECONDS >= deadline )); then | ||
| echo "::error::$ip:$port never became reachable" | ||
| tailscale ping -c 3 "$ip" || true | ||
| exit 1 | ||
| fi | ||
| sleep 10 | ||
| done | ||
| done | ||
| echo "✓ stack reachable at $ip" | ||
|
|
||
| - name: Forward the stack onto loopback | ||
| run: | | ||
| set -euo pipefail | ||
| # The suite reaches Bridge, the User Env controller, electrs and | ||
| # bitcoind as 127.0.0.1 from inside the Simulator, across five call | ||
| # sites in BitkitUITests/TrezorBridgeDashboardUITests.swift that are | ||
| # not all configurable. Relaying those ports keeps the test unchanged. | ||
| cat > /tmp/forward-stack.py <<'PY' | ||
| import asyncio | ||
| import sys | ||
|
|
||
| HOST = sys.argv[1] | ||
| PORTS = [int(port) for port in sys.argv[2:]] | ||
|
|
||
|
|
||
| async def pipe(reader, writer): | ||
| try: | ||
| while (chunk := await reader.read(65536)): | ||
| writer.write(chunk) | ||
| await writer.drain() | ||
| except Exception: | ||
| pass | ||
| finally: | ||
| writer.close() | ||
|
|
||
|
|
||
| def forward(port): | ||
| async def handle(local_reader, local_writer): | ||
| remote_reader, remote_writer = await asyncio.open_connection(HOST, port) | ||
| await asyncio.gather( | ||
| pipe(local_reader, remote_writer), | ||
| pipe(remote_reader, local_writer), | ||
| ) | ||
|
|
||
| return handle | ||
|
|
||
|
|
||
| async def main(): | ||
| servers = [ | ||
| await asyncio.start_server(forward(port), "127.0.0.1", port) | ||
| for port in PORTS | ||
| ] | ||
| print(f"forwarding {PORTS} to {HOST}", flush=True) | ||
| await asyncio.gather(*(server.serve_forever() for server in servers)) | ||
|
|
||
|
|
||
| asyncio.run(main()) | ||
| PY | ||
|
|
||
| nohup python3 /tmp/forward-stack.py "$STACK_IP" \ | ||
| 21325 9001 43782 60001 9002 6080 > /tmp/forward-stack.log 2>&1 & | ||
|
|
||
| # End to end, not just a local accept: the relay listens before it has | ||
| # dialled anything, so a port check against it passes either way. | ||
| deadline=$(( SECONDS + 120 )) | ||
| until curl -fsS -m 10 -X POST http://127.0.0.1:21325/enumerate | grep -q '"path"'; do | ||
| if (( SECONDS >= deadline )); then | ||
| echo "::error::bridge did not answer through the relay" | ||
| cat /tmp/forward-stack.log || true | ||
| exit 1 | ||
| fi | ||
| sleep 5 | ||
| done | ||
| curl -fsS -X POST http://127.0.0.1:21325/enumerate | ||
| echo "✓ bridge reachable on 127.0.0.1:21325" | ||
|
|
||
| - name: Boot simulator | ||
| env: | ||
| SIMULATOR_NAME: ${{ inputs.simulator_name }} | ||
| run: | | ||
| if ! xcodebuild -showsdks | grep -q "iOS Simulator"; then | ||
| xcodebuild -downloadPlatform iOS | ||
| fi | ||
|
|
||
| xcrun simctl shutdown all || true | ||
| xcrun simctl erase "$SIMULATOR_NAME" || true | ||
| defaults write com.apple.iphonesimulator DisableAllNotifications -bool true | ||
| xcrun simctl boot "$SIMULATOR_NAME" || true | ||
| xcrun simctl bootstatus "$SIMULATOR_NAME" -b | ||
| # First boot on a cold runner keeps working after bootstatus returns, | ||
| # and XCUITest attaching into that loses the first launch. | ||
| open -a Simulator | ||
| sleep 30 | ||
|
|
||
| - name: Run Trezor emulator UI tests | ||
| env: | ||
| TEST_TREZOR_EMU: "1" | ||
| TEST_TREZOR_RESET_STATE: "1" | ||
| TREZOR_BRIDGE: "true" | ||
| TREZOR_BRIDGE_URL: "http://127.0.0.1:21325" | ||
| TREZOR_ELECTRUM_URL: "tcp://127.0.0.1:60001" | ||
| E2E: "true" | ||
| E2E_BACKEND: "local" | ||
| E2E_NETWORK: "regtest" | ||
| GEO: "false" | ||
| SIMULATOR_NAME: ${{ inputs.simulator_name }} | ||
| SIMULATOR_OS: ${{ inputs.simulator_os }} | ||
| run: | | ||
| mkdir -p TestResults | ||
| set -o pipefail | ||
| xcodebuild test \ | ||
| -workspace Bitkit.xcodeproj/project.xcworkspace \ | ||
| -scheme BitkitAITests \ | ||
| -configuration Debug \ | ||
| -destination "platform=iOS Simulator,name=$SIMULATOR_NAME,OS=$SIMULATOR_OS" \ | ||
| -derivedDataPath DerivedData \ | ||
| -resultBundlePath TestResults/TrezorBridgeDashboardUITests.xcresult \ | ||
| SWIFT_ACTIVE_COMPILATION_CONDITIONS='DEBUG E2E_BUILD TEST_TREZOR_EMU' \ | ||
| -only-testing:BitkitUITests/TrezorBridgeDashboardUITests \ | ||
| -parallel-testing-enabled NO \ | ||
| -allowProvisioningUpdates \ | ||
| | xcbeautify | ||
|
|
||
| # xcbeautify prints one condensed line per failed test, which for an | ||
| # assertion carrying no message says only that it failed. The result | ||
| # bundle has the file and line, and the accessibility dump the UI tests | ||
| # attach to their messages. | ||
| - name: Report test failures | ||
| if: failure() | ||
| run: | | ||
| set -uo pipefail | ||
| bundle=TestResults/TrezorBridgeDashboardUITests.xcresult | ||
| if [ ! -d "$bundle" ]; then | ||
| echo "no result bundle at $bundle" | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "=== failures ===" | ||
| xcrun xcresulttool get test-results tests --path "$bundle" --format json 2>/dev/null \ | ||
| | jq -r '.. | objects | select(.nodeType? == "Failure Message") | .name' \ | ||
| || echo "could not read failure messages" | ||
|
|
||
| echo "=== summary ===" | ||
| xcrun xcresulttool get test-results summary --path "$bundle" --format json 2>/dev/null \ | ||
| | jq -r '.testFailures[]? | "\(.testName): \(.failureText)"' \ | ||
| || echo "could not read summary" | ||
|
|
||
| - name: Collect diagnostics | ||
| if: always() | ||
| run: | | ||
| mkdir -p ai-device-artifacts/trezor-remote | ||
| xcrun simctl io booted screenshot ai-device-artifacts/trezor-remote/simulator.png || true | ||
| if [ -d TestResults ]; then | ||
| cp -R TestResults ai-device-artifacts/trezor-remote/ || true | ||
| fi | ||
| curl --silent --show-error -X POST http://127.0.0.1:21325/enumerate > ai-device-artifacts/trezor-remote/bridge-enumerate.json 2>&1 || true | ||
| cp /tmp/forward-stack.log ai-device-artifacts/trezor-remote/ || true | ||
|
|
||
| - name: Upload diagnostics | ||
| if: always() | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: ai-device-tests-trezor-remote-${{ github.run_number }} | ||
| path: ai-device-artifacts/trezor-remote | ||
| if-no-files-found: warn | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When
bitkit-docker'smainbranch changes or is compromised before dispatch, this job executes that unreviewed revision after joining the project tailnet, exposing CI resources, reachable tailnet services, and the checkout-persisted GitHub token to arbitrary upstream code. Pin the checkout to a reviewed commit SHA. How this was verified: The workflow checks out mutablemain, joins Tailscale withTS_AUTHKEY, and then runsdocker composeand./scripts/trezor-emulatorfrom that checkout.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔵 TS_AUTHKEY now lands on GitHub-hosted runners — confirm it is ephemeral, tagged and ACL-scoped
The self-hosted
trezor-emupath consumes no secrets at all. Both new jobs put a Tailscale auth key on shared GitHub-hosted runners, and anything that can inject a step into this workflow (or any third-party action running in these jobs) can exfiltrate it and join your tailnet. Confirm TS_AUTHKEY is reusable-but-ephemeral, pre-authorized, and carries a dedicated tag whose ACL grants only the regtest/Trezor stack ports — not a general-purpose key. If it is not ephemeral, every run also leaves a dead node behind, which breaks theHostName ==peer lookup on the next run.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can’t confirm those properties from this PR. The workflow only references
${{ secrets.TS_AUTHKEY }}; GitHub does not expose whether that key is reusable, ephemeral, pre-authorized, or tagged, and the repository contains no Tailscale ACL configuration.The required configuration should be reusable (so both jobs can enroll), ephemeral (so each runner node is removed after logout/inactivity), pre-authorized (if device approval is enabled), and assigned a dedicated tag whose grants permit only the required regtest/Trezor endpoints. The ACL should not grant general tailnet access. A reusable but non-ephemeral key would leave nodes behind and make the
HostName ==peer lookup unreliable.Please verify the key settings and tag/grant policy in Tailscale administration, or use per-run OIDC/OAuth-generated ephemeral credentials. Until that is confirmed, this security concern remains unresolved.