threat-detection: GITHUB_STEP_SUMMARY is unwritable in the AWF sandbox, aborting the v0.4.0 detector with exit 2

[davidslater]

Created by GitHub Ace · View Session

Summary

With threat-detect v0.4.0, the external detection step aborts with exit 2 before the engine ever runs. Upstream #747 added an artifact-inventory table written to GITHUB_STEP_SUMMARY, and per spec TD-20c a failed write is a config error. Inside the AWF sandbox that write always fails, so:

stepsummary.WriteArtifactInventory → open failure
main.go → reason=config_error → exit 2
(no detection_result.json produced)
conclude → ERR_SYSTEM: Detection result file not found

Engine-agnostic, which is why all three upstream standalone smokes fail identically. Upstream has committed a resiliency fix on their side, but the wiring gap is ours and the inventory table is silently lost until it is fixed here.

This is the blocker for #50644 (the v0.4.0 pin bump) in practice.

Root cause

The detection step inherits GITHUB_STEP_SUMMARY=/home/runner/work/_temp/_runner_file_commands/step_summary_<uuid>. From sandbox/firewall/audit/docker-compose.redacted.yml on a real failing run:

environment:
  GITHUB_STEP_SUMMARY: /home/runner/work/_temp/_runner_file_commands/step_summary_7e854bcb-…
volumes:
  - /tmp:/tmp:rw
  - /home/runner/work/<repo>/<repo>:…:rw
  - /tmp/awf-…-chroot-home:/host/home/runner:rw   # empty home overlay

Two facts:

1. /home/runner/work/_temp is not mounted, and /home/runner is overlaid by an empty home volume. The runner's file-command path resolves into an empty dir inside the chroot → ENOENT.

   This is deliberate AWF isolation and must stay that way. If _runner_file_commands were mounted rw, a sandboxed agent could write GITHUB_ENV / GITHUB_OUTPUT / GITHUB_STEP_SUMMARY and inject directly into the host workflow. Mounting it would be a privilege-escalation hole. Do not "fix" this by adding a mount.

2. gh-aw's step-level GITHUB_STEP_SUMMARY override never reaches the process. The runner re-injects the real file-command variable for run: steps after step-level env: is applied, and AWF forwards the actual process env via --env-all. So the container sees the runner path despite the lock file setting the override.

Confirmed in this repo

The compiled Execute threat detection with AWF step in .github/workflows/ab-testing-advisor.lock.yml does carry the override:

GITHUB_STEP_SUMMARY: /tmp/gh-aw/agent-step-summary.md

(inherited from the copilot engine env block via extractStepEnvLines), and the AWF invocation uses --env-all with --mount /tmp/gh-aw/threat-detection:/tmp/gh-aw/threat-detection:rw. The override is present and ineffective, exactly as described.

A second, independent gap: even if the override did work, /tmp/gh-aw/agent-step-summary.md is the agent's summary path. The detection job neither touches it nor appends it — I checked the compiled detection: job and there is no Append agent step summary step and no touch. So the detector's inventory table would be written into a file nobody ever reads. Both halves need fixing.

Proposed fix

Mirror what the agent job already does (touch + append_agent_step_summary.sh), but with a detection-specific path inside the already-rw-mounted detection dir:

1. Add a ThreatDetectionStepSummaryPath constant = /tmp/gh-aw/threat-detection/step-summary.md.
2. touch it in the host-side PathSetup before the AWF invocation (same pattern as claude_engine.go:399).
3. Pass --step-summary /tmp/gh-aw/threat-detection/step-summary.md explicitly on the threat-detect command line, rather than relying on env. A flag is the correct mechanism here — the whole problem is that env for this variable is not controllable from a run: step, so any env-based fix is fragile by construction.
4. Add a post-execution step that appends the file to the real $GITHUB_STEP_SUMMARY on the host, after redaction, no-op when empty.
5. Add the path to buildUploadDetectionArtifactStep so it survives even if the append step is skipped.

The --step-summary flag exists as of upstream #739 and is supported by both threat-detect and threat-detect conclude.

Alternative considered and rejected: explicitly unsetting GITHUB_STEP_SUMMARY for the sandboxed step. That stops the exit-2 abort but permanently discards the inventory table and the detector's real prompt/verdict blocks, which is most of the value of #50653.

Note on conclude

threat-detect conclude runs on the host, not in the sandbox, so its --step-summary default ($GITHUB_STEP_SUMMARY) works correctly and needs no change. Worth an explicit test so this asymmetry is not "cleaned up" later by someone assuming both need the flag.

Related finding — the agent job's redirect is likely also ineffective

The same mechanism applies to the agent job. Every engine sets GITHUB_STEP_SUMMARY: AgentStepSummaryPath in its step env (claude_engine.go:480, copilot_engine_execution.go:532, codex_engine.go:366, gemini_engine.go:305, pi_engine.go:419) and append_agent_step_summary.sh appends /tmp/gh-aw/agent-step-summary.md afterwards.

If the runner re-injects the file-command variable there too, then agents have been writing to the real summary path — i.e. into the empty chroot overlay, failing silently — and agent-step-summary.md has been empty all along. Nobody noticed because, unlike the detector, no agent treats the write failure as an error.

This should be verified as part of this issue: check whether /tmp/gh-aw/agent-step-summary.md is non-empty on any recent agent run that produced step-summary output. If it is consistently empty, the same flag-or-wrapper fix is needed on the agent path, and it is worth splitting into its own issue since it touches all five engines.

Acceptance criteria

• External detection runs on v0.4.0 no longer exit 2 on the step-summary write.
• The detector's artifact-inventory table, rendered prompt, and verdict block appear in the job step summary.
• --step-summary passed explicitly to a path inside the rw-mounted ThreatDetectionDir; no new AWF mount added, and _runner_file_commands remains unmounted.
• Detection summary file touched before execution and appended after, no-op when empty.
• Path added to the detection artifact upload.
• Test asserting conclude is not given the sandbox path (it runs on the host).
• Verified whether the agent job's AgentStepSummaryPath redirect is also ineffective; follow-up issue filed if so.
• make recompile.

References

• Meta: Meta: enable gh-aw-threat-detection v0.4.0 on the external detection path #50656
• Overlaps the GITHUB_STEP_SUMMARY writability caution in threat-detection: stage prompt-template, prompt-import-tree, aw_info.json and comment-memory into the detection dir #50647 — that bullet should be removed in favour of this issue.
• Interacts with threat-detection: external path writes an unused prompt to the step summary, duplicating the detector's real one #50653 (which prompt renders into the summary) and threat-detection: bump external detector pin to threat-detect v0.4.* (Wait for v0.4.* to pass local smokes) #50644 (v0.4.0 pin).
• feat(artifacts): surface activation context and inventory gh-aw-threat-detection#747, feat(cli): add step-summary blocks for rendered prompt and verdict gh-aw-threat-detection#739