Skip to content

Publish auto-triage summaries when labels are queued first - #4992

Merged
paulmedynski merged 1 commit into
mainfrom
dev/priyankatiwari08/issue-backlog-triage
Nov 4, 2026
Merged

paulmedynski merged 1 commit into
mainfrom
dev/priyankatiwari08/issue-backlog-triage

Conversation

@priyankatiwari08

Copy link
Copy Markdown
Contributor

Description

The issue auto-triage workflow failed to publish completed triage summaries whenever the agent queued its Auto-Triage: Waiting for Author label change before its add_comment summary.

Root cause. gh-aw publishes safe outputs in queue order. The pre-publication validator (validate-triage-output.cjs) therefore rejected any batch with a label operation ahead of the comment (Triage cannot change labels before its summary comment.). This kept labels from changing without a visible summary, but it also discarded a complete, valid summary. In every failed run since Oct 14, the downloaded agent_output.json contained a finished summary with the label operation queued first:

Issue Run Queued items
#4969 41619094928 remove_labels, add_comment
#4965 41528557907 remove_labels, add_comment
#4953 41109525171 add_labels, add_comment
#4940 40746830015 remove_labels, add_comment
#4904 39789226820 remove_labels, add_comment
#4879 39494526187 remove_labels, add_comment
#4855 39032075412 remove_labels, add_comment

The prompt already asks for the comment first. The model does not follow that reliably, so the order is now fixed deterministically in trusted code instead.

Fix.

  • normalizeTriageOutput moves label operations queued before a single summary to just after it. All other items keep their relative order. Batches without exactly one summary are returned unchanged, and validation still rejects them.
  • validateTriageOutputFile normalizes and then validates. Only if validation passes does it write the reordered batch back to GH_AW_AGENT_OUTPUT. "Process Safe Outputs" reads that same file, so the summary is published before the label. A rejected batch leaves the file untouched and still fails the step, which blocks publication.
  • The workflow's validator step and the CLI entry point share this helper. The workflow was recompiled with gh aw compile (gh-aw v0.88.2); the lock change is limited to that step and the frontmatter hash, and a second compile is a no-op.
  • validateTriageOutput itself is unchanged and still rejects unnormalized label-first batches as a defense in depth.

No product code, public API, or package changes.

Issues

Restores triage summaries for the failure mode above. After merge, maintainers can comment /triage on #4969, #4965, #4953, #4940, #4904, #4879 and #4855 to regenerate their summaries.

Not addressed here: #4709 was skipped for a different reason. The workflow's daily AI-credit limit was exceeded (11,256 against a 5,000 threshold), so the agent never ran.

Testing

Guidelines

Please review the contribution guidelines before submitting a pull request:

gh-aw publishes safe outputs in queue order, so the triage validator
rejected the whole batch whenever the agent queued its Waiting for Author
label change before the summary comment. Seven recent issues lost their
completed triage summaries this way.

Move label operations after the single summary before validation, then
write the reordered batch back to the file that safe-output publication
reads. Rejected batches are left unchanged and still block publication.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the triage publication-ordering change.

No open issues 🎉

What changed in this PR

Fixes issue auto-triage publication when label changes are queued before a completed summary. The change normalizes publication order while preserving validation safeguards; no product code, public APIs, or packages change.

Changes:

  • Move early label operations after a single summary without mutating the input batch.
  • Share file normalization and validation between the workflow and CLI, rewriting only accepted batches.
  • Add regression coverage for ordering, rejected batches, file rewriting, and compiled workflow wiring.
File Description
.github/​workflows/​issue-triage.md Calls the shared file validator before publication.
.github/​workflows/​issue-triage.lock.yml Updates the generated workflow to match the validator integration.
.github/​scripts/​validate-triage-output.cjs Adds queue normalization and validation-gated file rewriting.
.github/​scripts/​tests/​validate-triage-output.test.cjs Adds normalization, file, CLI, and workflow-wiring regression tests.
.github/​scripts/​tests/​README.md Documents reordered publication and its test coverage.

Review effort: Balanced

@paulmedynski paulmedynski moved this from To triage to In review in SqlClient Board Nov 4, 2026
@paulmedynski
paulmedynski enabled auto-merge (squash) November 4, 2026 18:51
@paulmedynski
paulmedynski merged commit 025a3ea into main Nov 4, 2026
14 of 15 checks passed
@paulmedynski
paulmedynski deleted the dev/priyankatiwari08/issue-backlog-triage branch November 4, 2026 18:52
@github-project-automation github-project-automation Bot moved this from In review to Done in SqlClient Board Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants