Conversation
…count check, stronger exit protocol Addresses recommendations from the deep audit issue: - Rec 1: Add ABSOLUTE FINAL RULE block to both prompt and agent file - Rec 2: Scope make recompile to only run when .md files changed; abort if >50 files - Rec 3: Switch engine from copilot to claude with max-turns: 20 - Rec 4: Add file-count guard (>80 staged files → noop instead of oversized PR) - Rec 5: Secondary CI re-check at start of agent (handles self-healed CI) - Rec 6: Explicit warning against make deps-dev / make agent-finish - Regenerate hourly-ci-cleaner.lock.yml Agent-Logs-Url: https://github.com/github/gh-aw/sessions/caa147dd-be37-4692-b883-55a17607416b Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Updates the hourly-ci-cleaner automation to use the Claude engine with a hard turn limit, reduce oversized PRs from unnecessary workflow recompiles, and add explicit exit guardrails to avoid “no safe outputs” failures.
Changes:
- Switch workflow engine from Copilot to Claude with
max-turns: 20 - Add CI self-heal verification, conditional
make recompile, and staged-file count guardrails - Strengthen “must call safe-output tool” exit protocol in both workflow prompt and agent doc
Show a summary per file
| File | Description |
|---|---|
.github/workflows/hourly-ci-cleaner.md |
Updates the workflow prompt to Claude/max-turns and adds CI verification + recompile/PR size guardrails. |
.github/workflows/hourly-ci-cleaner.lock.yml |
Regenerates the locked workflow implementation to run Claude Code CLI and update engine/secrets/config. |
.github/agents/ci-cleaner.agent.md |
Aligns agent instructions with conditional recompile, file-count guardrails, and stricter exit protocol. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (2)
.github/workflows/hourly-ci-cleaner.lock.yml:737
PROMPT_TEXTis a shell variable but is not exported before invokingawf/bash -c. As a result,$PROMPT_TEXTinside the container will be empty, so Claude will run with an empty prompt. Export the variable (or write the prompt to a file and reference it) before theawfinvocation.
# Combine agent content with prompt
PROMPT_TEXT="$(printf '%s\n\n%s' "$AGENT_CONTENT" "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)")"
touch /tmp/gh-aw/agent-step-summary.md
# shellcheck disable=SC1003
sudo -E awf --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --tty --env-all --exclude-env ANTHROPIC_API_KEY --exclude-env GITHUB_MCP_SERVER_TOKEN --exclude-env MCP_GATEWAY_API_KEY --mount /opt/hostedtoolcache/go:/opt/hostedtoolcache/go:ro --mount /usr/bin/go:/usr/bin/go:ro --mount /usr/bin/make:/usr/bin/make:ro --mount /usr/local/bin/node:/usr/local/bin/node:ro --mount /usr/local/bin/npm:/usr/local/bin/npm:ro --mount /usr/local/lib/node_modules:/usr/local/lib/node_modules:ro --allow-domains '*.githubusercontent.com,anthropic.com,api.anthropic.com,api.github.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,cdn.playwright.dev,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,files.pythonhosted.org,ghcr.io,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.com,go.dev,golang.org,goproxy.io,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pkg.go.dev,playwright.download.prss.microsoft.com,ppa.launchpad.net,proxy.golang.org,pypi.org,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,sentry.io,statsig.anthropic.com,storage.googleapis.com,sum.golang.org,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com' --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --image-tag 0.25.13 --skip-pull --enable-api-proxy \
-- /bin/bash -c 'export PATH="$(find /opt/hostedtoolcache -maxdepth 4 -type d -name bin 2>/dev/null | tr '\''\n'\'' '\'':'\'')$PATH"; [ -n "$GOROOT" ] && export PATH="$GOROOT/bin:$PATH" || true && claude --print --disable-slash-commands --no-chrome --max-turns 20 --mcp-config /tmp/gh-aw/mcp-config/mcp-servers.json --allowed-tools Bash,BashOutput,Edit,ExitPlanMode,Glob,Grep,KillBash,LS,MultiEdit,NotebookEdit,NotebookRead,Read,Task,TodoWrite,Write,mcp__github__download_workflow_run_artifact,mcp__github__get_code_scanning_alert,mcp__github__get_commit,mcp__github__get_dependabot_alert,mcp__github__get_discussion,mcp__github__get_discussion_comments,mcp__github__get_file_contents,mcp__github__get_job_logs,mcp__github__get_label,mcp__github__get_latest_release,mcp__github__get_me,mcp__github__get_notification_details,mcp__github__get_pull_request,mcp__github__get_pull_request_comments,mcp__github__get_pull_request_diff,mcp__github__get_pull_request_files,mcp__github__get_pull_request_review_comments,mcp__github__get_pull_request_reviews,mcp__github__get_pull_request_status,mcp__github__get_release_by_tag,mcp__github__get_secret_scanning_alert,mcp__github__get_tag,mcp__github__get_workflow_run,mcp__github__get_workflow_run_logs,mcp__github__get_workflow_run_usage,mcp__github__issue_read,mcp__github__list_branches,mcp__github__list_code_scanning_alerts,mcp__github__list_commits,mcp__github__list_dependabot_alerts,mcp__github__list_discussion_categories,mcp__github__list_discussions,mcp__github__list_issue_types,mcp__github__list_issues,mcp__github__list_label,mcp__github__list_notifications,mcp__github__list_pull_requests,mcp__github__list_releases,mcp__github__list_secret_scanning_alerts,mcp__github__list_starred_repositories,mcp__github__list_tags,mcp__github__list_workflow_jobs,mcp__github__list_workflow_run_artifacts,mcp__github__list_workflow_runs,mcp__github__list_workflows,mcp__github__pull_request_read,mcp__github__search_code,mcp__github__search_issues,mcp__github__search_orgs,mcp__github__search_pull_requests,mcp__github__search_repositories,mcp__github__search_users --debug-file /tmp/gh-aw/agent-stdio.log --verbose --permission-mode bypassPermissions --output-format stream-json "$PROMPT_TEXT"${GH_AW_MODEL_AGENT_CLAUDE:+ --model "$GH_AW_MODEL_AGENT_CLAUDE"}' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log
.github/workflows/hourly-ci-cleaner.lock.yml:737
--debug-file /tmp/gh-aw/agent-stdio.logwrites to the same file that stdout/stderr is being appended to viatee -a /tmp/gh-aw/agent-stdio.log. Concurrent writes will interleave and can break downstream log parsing. Use a separate debug file, or stop teeing into the same path.
-- /bin/bash -c 'export PATH="$(find /opt/hostedtoolcache -maxdepth 4 -type d -name bin 2>/dev/null | tr '\''\n'\'' '\'':'\'')$PATH"; [ -n "$GOROOT" ] && export PATH="$GOROOT/bin:$PATH" || true && claude --print --disable-slash-commands --no-chrome --max-turns 20 --mcp-config /tmp/gh-aw/mcp-config/mcp-servers.json --allowed-tools Bash,BashOutput,Edit,ExitPlanMode,Glob,Grep,KillBash,LS,MultiEdit,NotebookEdit,NotebookRead,Read,Task,TodoWrite,Write,mcp__github__download_workflow_run_artifact,mcp__github__get_code_scanning_alert,mcp__github__get_commit,mcp__github__get_dependabot_alert,mcp__github__get_discussion,mcp__github__get_discussion_comments,mcp__github__get_file_contents,mcp__github__get_job_logs,mcp__github__get_label,mcp__github__get_latest_release,mcp__github__get_me,mcp__github__get_notification_details,mcp__github__get_pull_request,mcp__github__get_pull_request_comments,mcp__github__get_pull_request_diff,mcp__github__get_pull_request_files,mcp__github__get_pull_request_review_comments,mcp__github__get_pull_request_reviews,mcp__github__get_pull_request_status,mcp__github__get_release_by_tag,mcp__github__get_secret_scanning_alert,mcp__github__get_tag,mcp__github__get_workflow_run,mcp__github__get_workflow_run_logs,mcp__github__get_workflow_run_usage,mcp__github__issue_read,mcp__github__list_branches,mcp__github__list_code_scanning_alerts,mcp__github__list_commits,mcp__github__list_dependabot_alerts,mcp__github__list_discussion_categories,mcp__github__list_discussions,mcp__github__list_issue_types,mcp__github__list_issues,mcp__github__list_label,mcp__github__list_notifications,mcp__github__list_pull_requests,mcp__github__list_releases,mcp__github__list_secret_scanning_alerts,mcp__github__list_starred_repositories,mcp__github__list_tags,mcp__github__list_workflow_jobs,mcp__github__list_workflow_run_artifacts,mcp__github__list_workflow_runs,mcp__github__list_workflows,mcp__github__pull_request_read,mcp__github__search_code,mcp__github__search_issues,mcp__github__search_orgs,mcp__github__search_pull_requests,mcp__github__search_repositories,mcp__github__search_users --debug-file /tmp/gh-aw/agent-stdio.log --verbose --permission-mode bypassPermissions --output-format stream-json "$PROMPT_TEXT"${GH_AW_MODEL_AGENT_CLAUDE:+ --model "$GH_AW_MODEL_AGENT_CLAUDE"}' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log
- Files reviewed: 3/3 changed files
- Comments generated: 4
| git diff --name-only | grep '\.md$' | ||
| ``` | ||
| 2. **If NO `.md` files changed** → **SKIP `make recompile` entirely**. | ||
| 3. **If `.md` files changed** → Run `make recompile`, then immediately check: |
There was a problem hiding this comment.
This check is described as “workflow .md files”, but grep '\.md$' will match any Markdown file anywhere in the repo (docs, READMEs, etc.), causing unnecessary make recompile runs and the large-diff problem this guard is meant to prevent. Consider scoping the pattern to workflow markdown paths (e.g., .github/workflows/*.md).
| git diff --name-only | grep '\.md$' | |
| ``` | |
| 2. **If NO `.md` files changed** → **SKIP `make recompile` entirely**. | |
| 3. **If `.md` files changed** → Run `make recompile`, then immediately check: | |
| git diff --name-only | grep '^\.github/workflows/.*\.md$' |
- If NO workflow
.mdfiles changed → SKIPmake recompileentirely. - If workflow
.mdfiles changed → Runmake recompile, then immediately check:
| **Before running recompile**, check whether any workflow `.md` files were modified: | ||
|
|
||
| ```bash | ||
| git diff --name-only | grep '\.md$' | ||
| ``` |
There was a problem hiding this comment.
This is intended to detect changes to workflow Markdown files, but git diff --name-only | grep '\.md$' will match any .md file in the repo (including unrelated docs). That can trigger unnecessary make recompile runs and recreate the “PR too large” failure mode. Narrow the match to workflow markdown paths (e.g., .github/workflows/*.md).
|
@copilot apply changes based on the comments in this thread |
Previously grep '\.md$' matched any markdown file in the repo (docs, READMEs, etc.), which could trigger unnecessary make recompile runs and recreate the E003 "PR too large" failure. Narrowed to '^\.github/workflows/.*\.md$' in both the workflow prompt and agent file. Agent-Logs-Url: https://github.com/github/gh-aw/sessions/31e0681e-fef5-4712-8a1a-d3dd8374f40a Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Applied in d1a2694. The Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
The
hourly-ci-cleanerworkflow had a ~43% failure rate, dominated by "no safe outputs" (44% of failures) and oversized PRs from unconditionalmake recompileruns (166 files in one case).Engine
copilot→claudewithmax-turns: 20, providing the hard turn limit Copilot doesn't supportCI Status Check
gh run list --limit=2before doing any work — handles the case where CI self-healed between thecheck_ci_statusjob and agent executionfailureRecompile Guard (was causing E003 "PR too large")
Only run
make recompilewhen.mdworkflow files actually changed:After recompile, abort with
noopif >50 files changed (signals binary/template mismatch).File-Count Guard
Before creating a PR, check total staged files — call
noopinstead if >80:Exit Protocol
Added an ABSOLUTE FINAL RULE block (in both the workflow prompt and the agent file) with unconditional language: if the response is about to end without a safe-output tool call, call
noopimmediately.Deps Warning
Moved the
make deps-dev/make agent-finishprohibition to be immediately visible under "Your Task" — both commands take 10–15 min and deps are already installed by setup steps.Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
https://api.github.com/graphql/usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw(http block)/usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw cXU4ZOQ/cBxinx7a-1(http block)/usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw ase,codename,all-Dcodeql.chainer.v2=/tmp/codeql.8AmckaiT(http block)https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b/usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha(http block)/usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha get --global docker-buildx http.https://gitgit(http block)/usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha --get-regexp --global rgo/bin/git(http block)https://api.github.com/repos/github/gh-aw/usr/bin/gh gh api /repos/github/gh-aw --jq .visibility(http block)/usr/bin/gh gh api /repos/github/gh-aw --jq .visibility --local committer.email r: $owner, name: $name) { hasDiscussionsEnabled } }(http block)https://api.github.com/repos/githubnext/agentics/git/ref/tags//usr/bin/gh gh api /repos/githubnext/agentics/git/ref/tags/# --jq .object.sha(http block)/usr/bin/gh gh api /repos/githubnext/agentics/git/ref/tags/# --jq .object.sha get --local $name) { hasDiscussionsEnabled } } user.email(http block)If you need me to access, download, or install something from one of these locations, you can either: