Add ci-status Copilot skill for PR build investigation#11024
Open
simonrozsival wants to merge 9 commits intomainfrom
Open
Add ci-status Copilot skill for PR build investigation#11024simonrozsival wants to merge 9 commits intomainfrom
simonrozsival wants to merge 9 commits intomainfrom
Conversation
The skill was not being invoked when users asked "check CI status" or "why is CI failing" because the description focused on AZDO URLs and .binlog analysis. The LLM would check gh pr checks first (which shows all green) and never reach the skill. Changes: - SKILL.md: Rewrite description to lead with common trigger phrases (CI status, CI failures, PR blocked) and emphasize this is the PRIMARY CI tool since the real CI runs on Azure DevOps - copilot-instructions.md: Add "CI / Build Investigation" section explaining the Azure DevOps CI architecture and that the skill must always be invoked first Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Updates Copilot guidance so CI-related user queries trigger the azdo-build-investigator skill (to surface internal Azure DevOps failures that may not appear in GitHub checks).
Changes:
- Rewrites the
azdo-build-investigatorskill description to emphasize CI-status/failure trigger phrases and that AZDO is the primary CI signal. - Adds a new “CI / Build Investigation” section to
.github/copilot-instructions.mdinstructing Copilot to invokeazdo-build-investigatorfirst for CI questions.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| .github/skills/azdo-build-investigator/SKILL.md | Strengthens trigger phrasing and emphasizes AZDO as the authoritative CI source. |
| .github/copilot-instructions.md | Documents AZDO-centric CI architecture and directs Copilot to start investigations with the AZDO skill. |
…ce files New skill that serves as the primary entry point for CI status queries: - Auto-detects current PR from git branch (no URL needed) - Phase 1: Quick summary of GitHub checks + AZDO build status - Phase 2: Deep investigation (logs, binlogs) only on request - Reference files loaded conditionally (error-patterns, binlog-analysis) - Both bash and PowerShell examples throughout - Clear boundaries, error handling, and output format spec - copilot-instructions.md updated to point to ci-status as primary Inspired by DataDog/dd-trace-dotnet analyze-azdo-build skill patterns and Anthropic skill creation best practices. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The CI investigation improvements are now in the dedicated ci-status skill and copilot-instructions.md instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
simonrozsival
changed the title
Add Step 3b to query AZDO test runs API for already-published test failures even when the overall build is still running. Update the output format to show failed test names and error messages prominently so they can be investigated immediately. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Show job progress counters (completed/running/waiting), elapsed time, and estimated completion based on median duration of last 5 successful runs of the same pipeline definition. Also handles overdue builds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Show separate progress counters and ETA for each AZDO build (public dotnet-android and internal Xamarin.Android-PR). Each pipeline has its own definition ID and typical duration, so ETAs are computed independently from the last 5 successful runs of each. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Label builds by their definitionName (e.g. 'dotnet-android', 'Xamarin.Android-PR') instead of misleading 'Public'/'Internal'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The two AZDO pipelines (dotnet-android and Xamarin.Android-PR) run independently — neither waits for the other. Document that Xamarin.Android-PR's PR trigger is configured in AZDO UI (not YAML) and may take a few minutes to start. Handle the GitHub 'Expected' check state explicitly instead of vaguely saying 'not found'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Detect fork vs direct PRs via isCrossRepository. Fork PRs get full dotnet-android pipeline (with tests) but no Xamarin.Android-PR. Direct PRs get build-only dotnet-android and full Xamarin.Android-PR. ETA computation filters historical builds by duration profile to match the current PR type. Output shows fork status prominently. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
Status report example: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a dedicated
ci-statusCopilot skill that checks both GitHub checks and internal Azure DevOps builds when investigating CI status on a PR. Also adds a CI/Build Investigation section tocopilot-instructions.mdso the agent always uses the skill instead of relying ongh pr checksalone (which misses the internal AZDO builds).Changes
.github/skills/ci-status/SKILL.md— new skill with auto-detect PR from git branch, two-phase workflow (GitHub checks → AZDO timeline), and structured output.github/skills/ci-status/references/— reference docs for binlog analysis and common error patterns.github/copilot-instructions.md— added CI / Build Investigation section directing the agent to useci-statusfirst, thenazdo-build-investigatorfor deep .binlog analysis