From a5de46438b56414edbd64022567e36cc2a1c56b5 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sun, 2 Aug 2026 14:43:14 -0700 Subject: [PATCH 1/9] Component PR owners merge their own release notes PR Reviewing and approving a component release notes PR is not enough - the owner is responsible for merging it into the milestone base branch once it's ready, rather than leaving it for someone else. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a851ab96-8780-407d-97d0-50eed422ead5 --- .github/skills/release-notes/references/pr-layout.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/.github/skills/release-notes/references/pr-layout.md b/.github/skills/release-notes/references/pr-layout.md index 23efdd4f6f3..55c84d56f6b 100644 --- a/.github/skills/release-notes/references/pr-layout.md +++ b/.github/skills/release-notes/references/pr-layout.md @@ -19,6 +19,8 @@ The set of components and their release notes files is defined in [`component-ma Each component PR merges into the base branch. When all component PRs merge, the base PR's diff is the full milestone — there is no separate consolidation PR. +**The component owner merges their own PR.** Reviewing and approving is not enough — the owner is responsible for merging their component PR into the base branch once it's ready, rather than leaving it for someone else to merge. + ## PR title convention - Base PR: `[release-notes] .NET {version} {milestone-label}` (e.g. `[release-notes] .NET 11 Preview 4`). @@ -26,4 +28,6 @@ Each component PR merges into the base branch. When all component PRs merge, the ## Draft state -Open both the base PR and every component PR as **drafts** (`gh pr create --draft`). Component teams promote their PR to ready-for-review once they've vetted the AI-authored content (including any `` placeholders). The base PR stays a draft until the milestone ships. +Open both the base PR and every component PR as **drafts** (`gh pr create --draft`). Component teams promote their PR to ready-for-review once they've vetted the AI-authored content (including any `` placeholders), then merge it into the base branch themselves. The base PR stays a draft until the milestone ships. + +When notifying owners that their PRs are open, state that they own the merge, not just the review. From 7c4c69b6b02e8dcf3dfef338833594d5ea213e75 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sun, 2 Aug 2026 15:05:38 -0700 Subject: [PATCH 2/9] Encode .NET 11 Preview 7 retrospective into the release-notes skills Add a validate-code-samples skill and fold the lessons from generating the Preview 7 notes back into the surrounding skills. - Add .github/skills/validate-code-samples/SKILL.md, the runtime-verification stage that api-diff-validation already referenced but that did not exist. Covers acquiring the milestone build from ci.dot.net (build-metadata.json already emits sdk_url), pinning provenance via productCommit, installing scoped rather than machine-wide, and building/running every documented claim. Static API checks cannot see JS surface area, cannot catch inverted polarity, and cannot catch runtime-only requirements. - generate-changes: add a milestone cross-check with measured per-repo coverage. Only aspnetcore, runtime, sdk, and (weakly) efcore populate preview milestones reliably; roslyn and razor have no 11.x milestones at all. The milestone supplements changes.json, never replaces it. - api-verification: add the milestone-provenance rule. A change that shipped in an earlier preview still verifies against the current build, so presence in the build does not prove it belongs in these notes. - format-template: add the "Upgrading from the previous preview" section so preview-to-preview breaking changes are documented. - pr-layout: document creating the base branch first (branch globs exclude it), verifying assignees actually stuck, and the gh pr edit REST workaround. - component-mapping: add an owner-freshness check using the assignees endpoint. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a851ab96-8780-407d-97d0-50eed422ead5 --- .github/skills/api-diff-validation/SKILL.md | 4 +- .github/skills/generate-changes/SKILL.md | 36 +++++ .github/skills/release-notes/SKILL.md | 10 +- .../references/api-verification.md | 38 +++++ .../references/component-mapping.md | 17 +++ .../references/format-template.md | 28 ++++ .../release-notes/references/pr-layout.md | 25 +++ .github/skills/validate-code-samples/SKILL.md | 143 ++++++++++++++++++ 8 files changed, 295 insertions(+), 6 deletions(-) create mode 100644 .github/skills/validate-code-samples/SKILL.md diff --git a/.github/skills/api-diff-validation/SKILL.md b/.github/skills/api-diff-validation/SKILL.md index 6cc13f6e6c5..771c9d54fe4 100644 --- a/.github/skills/api-diff-validation/SKILL.md +++ b/.github/skills/api-diff-validation/SKILL.md @@ -38,8 +38,8 @@ the real ref packs from the build's feed, so it answers "does this API exist in Read `build-metadata.json` for the target. Use its `nuget.source` as `$FEED` and the ref-pack versions (e.g. `Microsoft.NETCore.App.Ref@`) as `$VER`. For behavior that requires the SDK (rare here), install it side-by-side and scoped — -see the `validate-code-samples` skill. Do **not** trust a machine-wide SDK for -preview work. +see the [`validate-code-samples`](../validate-code-samples/SKILL.md) skill. Do +**not** trust a machine-wide SDK for preview work. ## 2. Verify APIs exist with dotnet-inspect diff --git a/.github/skills/generate-changes/SKILL.md b/.github/skills/generate-changes/SKILL.md index da068bceebb..a8b83c9b460 100644 --- a/.github/skills/generate-changes/SKILL.md +++ b/.github/skills/generate-changes/SKILL.md @@ -126,4 +126,40 @@ The output file must follow the shared schema documented in [changes-schema.md]( - stable `id` values in `repo@shortcommit` format - same authoritative source of truth used by later skills +## Milestone cross-check + +`changes.json` is derived from a VMR source-manifest diff. That makes it authoritative for *what +flowed into the build*, but it is a commit-shaped view, and a feature can be easy to overlook in it. +For the repos that maintain preview milestones, sweep the milestone as a **second, independent view** +of the same release and reconcile anything that looks like a user-facing feature but never made it +into the notes. + +```bash +gh api -X GET search/issues \ + -f q="repo:dotnet/aspnetcore is:pr is:merged milestone:11.0-preview7" \ + --jq '.total_count' +``` + +Milestone discipline varies by repo, so this check only applies where it is actually maintained. +Measured for 11.0-preview7: + +| Repo | Merged PRs in milestone | Entries in `changes.json` | Use it? | +| ---- | ----------------------- | ------------------------- | ------- | +| `dotnet/sdk` | 177 | 199 | Yes | +| `dotnet/aspnetcore` | 176 | 205 | Yes | +| `dotnet/runtime` | 583 | 692 | Yes | +| `dotnet/efcore` | 77 | 107 | Yes, with weaker coverage | +| `dotnet/roslyn`, `dotnet/razor` | no `11.x` milestones exist | 108 (roslyn) | No | + +Rules for using it: + +- **Supplement, never replace.** `changes.json` stays the source of truth. The milestone is a + prompt to go back and look, not an alternative manifest. +- **The milestone is a subset.** It excludes infrastructure and dependency-flow PRs that + legitimately appear in `changes.json`, so the counts will not match and are not meant to. +- **Only the missing direction matters.** What is worth acting on is a PR in the milestone that + describes a user-facing change and has no corresponding entry in the notes. +- **Confirm the milestone exists before relying on its absence.** Repos without `11.x` milestones + will return zero results, which means "not tracked here", not "nothing shipped". + Once `changes.json` exists, the next step is usually `generate-features`. diff --git a/.github/skills/release-notes/SKILL.md b/.github/skills/release-notes/SKILL.md index 15368922ea3..ada31355ea7 100644 --- a/.github/skills/release-notes/SKILL.md +++ b/.github/skills/release-notes/SKILL.md @@ -1,7 +1,7 @@ --- name: release-notes -description: Generate and maintain .NET release notes from `features.json`. Uses `generate-changes` for authoritative shipped-change data, `generate-features` for scoring/triage, `update-existing-branch` for incremental reruns on populated branches, `editorial-scoring` for the shared rubric, `api-diff` to generate API diff reports and `api-diff-validation` for API verification, and a multi-model `review-release-notes` pass for final editorial QA. -compatibility: Requires GitHub MCP server or gh CLI for cross-repo queries. Pairs with the generate-changes, generate-features, update-existing-branch, editorial-scoring, api-diff, api-diff-validation, and review-release-notes skills. Claude Opus 4.6 is the default workflow model; the preferred final reviewer pair is Claude Opus 4.6 + GPT-5.4 for broader editorial feedback. +description: Generate and maintain .NET release notes from `features.json`. Uses `generate-changes` for authoritative shipped-change data, `generate-features` for scoring/triage, `update-existing-branch` for incremental reruns on populated branches, `editorial-scoring` for the shared rubric, `api-diff` to generate API diff reports, `api-diff-validation` for API verification, `validate-code-samples` to build and run the documented claims against the milestone build, and a multi-model `review-release-notes` pass for final editorial QA. +compatibility: Requires GitHub MCP server or gh CLI for cross-repo queries. Pairs with the generate-changes, generate-features, update-existing-branch, editorial-scoring, api-diff, api-diff-validation, validate-code-samples, and review-release-notes skills. Claude Opus 4.6 is the default workflow model; the preferred final reviewer pair is Claude Opus 4.6 + GPT-5.4 for broader editorial feedback. --- # .NET Release Notes @@ -17,8 +17,9 @@ This skill is the **editorial writing stage** of the pipeline. It turns a scored 3. `update-existing-branch` handles incremental reruns when a milestone branch already exists, merging deltas instead of restarting from scratch 4. `api-diff-validation` / `dotnet-inspect` verifies public APIs and confirms suspect features still exist in the shipped build 5. `release-notes` writes curated markdown using the higher-value entries from `features.json` -6. `review-release-notes` runs a final multi-model editorial QA pass against the scoring rubric and examples -7. Output is a set of pull requests per release milestone in dotnet/core: a base PR that holds shared metadata (`changes.json`, `features.json`, `README.md`, `build-metadata.json`) and one PR per component file. Each component PR targets the base branch so component teams review and edit their file in isolation. See [`pr-layout.md`](references/pr-layout.md) for the full layout and naming scheme. +6. `validate-code-samples` builds and runs the documented claims against the milestone build, catching what static API verification cannot see +7. `review-release-notes` runs a final multi-model editorial QA pass against the scoring rubric and examples +8. Output is a set of pull requests per release milestone in dotnet/core: a base PR that holds shared metadata (`changes.json`, `features.json`, `README.md`, `build-metadata.json`) and one PR per component file. Each component PR targets the base branch so component teams review and edit their file in isolation. See [`pr-layout.md`](references/pr-layout.md) for the full layout and naming scheme. ## Local testing (no PRs) @@ -45,4 +46,5 @@ handling review comments without clobbering human edits. - [format-template.md](references/format-template.md) — markdown document structure - [editorial-rules.md](references/editorial-rules.md) — tone, attribution, naming - [api-verification.md](references/api-verification.md) — using dotnet-inspect to verify APIs +- [../validate-code-samples/SKILL.md](../validate-code-samples/SKILL.md) — building and running the documented claims against the milestone build - [examples/](references/examples/) — curated examples from previous releases, organized by component. **Read the examples for your component before writing.** The [examples/README.md](references/examples/README.md) lists 12 editorial principles derived from what works and what doesn't in past release notes. diff --git a/.github/skills/release-notes/references/api-verification.md b/.github/skills/release-notes/references/api-verification.md index 3aee81934f1..93ec505ec61 100644 --- a/.github/skills/release-notes/references/api-verification.md +++ b/.github/skills/release-notes/references/api-verification.md @@ -116,6 +116,32 @@ You do NOT need to verify: - General concepts (e.g., "Zstandard compression" as a concept vs `ZstandardCompressionProvider` as a type) - CLI flags (e.g., `dotnet test --artifacts-path`) +## Verify the change shipped in *this* milestone + +API verification answers "does this exist in the build". It does not answer "did this change in this +release" — and those come apart in a way that is easy to miss. + +A behavior observed in the Preview 7 build was traced to a fix that shipped in **Preview 3**. It was +real, it was verifiable, and documenting it in the Preview 7 notes would have been wrong. Anything +still present from an earlier release will verify perfectly against the current build. + +The rule: **every documented change must trace to a PR in this milestone**, not merely be observably +true in the build. + +1. **`changes.json` membership is the primary test.** If a change has no entry, it did not flow into + this milestone. Confirm before writing it up. +2. **Check the PR's merge date and milestone** where the repo maintains them — + `dotnet/aspnetcore`, `dotnet/runtime`, `dotnet/sdk`, and `dotnet/efcore` do; + `dotnet/roslyn` and `dotnet/razor` do not (see + [`generate-changes`](../../generate-changes/SKILL.md)). For repos without milestones, + `changes.json` membership is the only available provenance signal. +3. **Be suspicious of anything discovered by testing rather than from `changes.json`.** Finding a + nice behavior while validating samples is a good way to find *previous* releases' features. Trace + it to a PR in this milestone before promoting it. + +This is the mirror image of the revert check below: a revert means something in `changes.json` is not +in the build, and stale provenance means something in the build is not in `changes.json`. + ## What to do when verification fails If `dotnet-inspect` can't find a type: @@ -128,4 +154,16 @@ If `dotnet-inspect` can't find a type: When in doubt, describe the feature without naming specific types and link to the PR. A correct prose description is always better than a wrong code sample. +## Static verification is not enough + +Everything above confirms that a *managed symbol exists*. Three important classes of error survive it: + +- **JavaScript and browser-facing APIs** are invisible to `dotnet-inspect` entirely. +- **Defaults and polarity** are not checked by a name lookup. A rename from `EnableX` to `DisableX` + passes an existence check while inverting the meaning of the documented claim. +- **Runtime behavior** — a sample can compile and still fail on request. + +After the notes are drafted, run [`validate-code-samples`](../../validate-code-samples/SKILL.md) to +build and execute the documented claims against the milestone build. + For scoring and feature selection, this is also a **quality bar**: if a change only looks interesting because it seems to add a new API, but you cannot identify that API in the public surface, score it down sharply. That usually means it is internal plumbing, a refactor, or an existing niche surface getting maintenance rather than a real release-note feature. diff --git a/.github/skills/release-notes/references/component-mapping.md b/.github/skills/release-notes/references/component-mapping.md index 37d7ae8c79e..17e4ecf3c63 100644 --- a/.github/skills/release-notes/references/component-mapping.md +++ b/.github/skills/release-notes/references/component-mapping.md @@ -24,6 +24,23 @@ Each release notes file gets its own per-component branch named `release-notes/{ The agent assigns each component PR to its default assignee(s) when opening the PR (`gh pr create --assignee ...`), so the right team sees it in their review queue. +### Check that the owners are still current + +This table is maintained by hand and goes stale as people change roles. In a single milestone, three +of the rows were wrong: one owner had retired, one had moved off the component, and one component had +been handed to someone new. + +Before opening the PRs, confirm each assignee can actually be assigned: + +```bash +gh api repos/dotnet/core/assignees/ --silent # exit 0 = assignable, 404 = not +``` + +A 404 means the mapping is stale, not that the call failed — GitHub will accept the PR creation and +silently drop that assignee. Treat it as a prompt to confirm the current owner and update this file, +rather than working around it. See [`pr-layout.md`](pr-layout.md) for the verification step after +the PRs are created. + ### Components contributed out-of-band (not in the VMR) These components ship with .NET but live outside the VMR, so `changes.json` won't contain entries for them. The agent still creates a stub PR for each so the component team can push their own content (or close the PR if there is nothing noteworthy this milestone). diff --git a/.github/skills/release-notes/references/format-template.md b/.github/skills/release-notes/references/format-template.md index 0e8618e6c62..a2ff012bc6b 100644 --- a/.github/skills/release-notes/references/format-template.md +++ b/.github/skills/release-notes/references/format-template.md @@ -89,6 +89,34 @@ Known component docs links: 4. **Feature ordering** — highest customer impact first 5. **Breaking changes near the end** — low-score entries with `breaking_changes: true` usually belong in a short section before Bug fixes, not as full feature sections 6. **Preview feature callout** — when a feature is listed in `release-notes/features.json`, start its section with the standard blockquote callout from that file +7. **Call out what breaks an upgrade from the previous preview** — see below + +## Upgrading from the previous preview + +Preview notes are read mostly by people who are already running the *previous* preview. For them the +most valuable content is often not the new feature, but the thing that stops their existing code +from building or working after they move to this build. + +Cover these when they apply. They tend to surface only when an existing project is actually upgraded +to the new build, which is one of the reasons +[`validate-code-samples`](../../validate-code-samples/SKILL.md) runs against a maintained sample set: + +- **Renamed APIs**, especially renames that invert meaning (`EnableX` becoming `DisableX`). Show the + before and after, and state the new default explicitly. +- **New analyzer diagnostics that fire on previously clean code.** Code that built without warnings + on the last preview and now reports diagnostics is an upgrade issue even though nothing in the + user's code changed. Name the diagnostic IDs. +- **Removed or replaced workarounds.** If a bug that required a workaround is now fixed, say so, so + users can delete the workaround rather than carrying it forward. +- **Changed defaults**, where existing code keeps compiling but behaves differently. + +Do not turn this into a changelog of everything that moved. Include an item only when a user +upgrading from the previous preview would otherwise hit a build error, a new warning, or a silent +behavior change. + +Attribute these to the milestone that actually changed them. A fix that shipped two previews ago is +not upgrade guidance for this one — see +[api-verification.md](api-verification.md) for the provenance rule. ## Issue and PR references diff --git a/.github/skills/release-notes/references/pr-layout.md b/.github/skills/release-notes/references/pr-layout.md index 55c84d56f6b..39c355f7e5e 100644 --- a/.github/skills/release-notes/references/pr-layout.md +++ b/.github/skills/release-notes/references/pr-layout.md @@ -15,6 +15,31 @@ The set of components and their release notes files is defined in [`component-ma - Each `{component}.md` lives on its **matching component branch only**. The agent never edits another component's file from the wrong branch. - The milestone landing page `{version}.md` (for example, `11.0.0-preview.4.md`) is **not produced by this skill**. The .NET release team generates it through separate artifacts-publishing automation, so the agent leaves it alone on every branch. +## Creating the PRs + +Order matters, and two of these steps fail silently. + +1. **Push the base branch first.** Component PRs target it, so it must exist on the remote before + any component PR can be opened. Note that a glob like `release-notes/{version}-{slug}-*` matches + the component branches but **not** the base branch — verifying with that pattern reports success + while the base branch is still local-only. +2. **Confirm each component branch's parent is the base branch commit.** When it is, each component + PR shows a clean one-file diff instead of also restating the shared metadata. +3. **Open the base PR against `main`**, then the component PRs against the base branch. +4. **Verify assignees after creating each PR.** GitHub silently drops assignees who lack access to + the repo: the API returns success and the PR is created with the assignee missing. Re-read the PR + and compare against the intended list rather than trusting the exit code. + +### gh pr edit does not work for assignees on this repo + +`gh pr edit --add-assignee` / `--remove-assignee` fails against `dotnet/core` with a Projects +(classic) GraphQL deprecation error, leaving assignees unchanged. Use the REST endpoints: + +```bash +gh api -X POST repos/dotnet/core/issues/{number}/assignees -f "assignees[]=" +gh api -X DELETE repos/dotnet/core/issues/{number}/assignees -f "assignees[]=" +``` + ## Merge flow Each component PR merges into the base branch. When all component PRs merge, the base PR's diff is the full milestone — there is no separate consolidation PR. diff --git a/.github/skills/validate-code-samples/SKILL.md b/.github/skills/validate-code-samples/SKILL.md new file mode 100644 index 00000000000..a047ec6c92c --- /dev/null +++ b/.github/skills/validate-code-samples/SKILL.md @@ -0,0 +1,143 @@ +--- +name: validate-code-samples +description: Verify release notes claims by building and running them against the actual .NET build for the milestone. Covers acquiring a scoped SDK from ci.dot.net, exercising every documented API and code sample, and catching the errors that static API verification cannot see - non-existent JavaScript APIs, inverted defaults, and runtime failures. USE FOR - validating a drafted component's release notes before the PR goes up, checking that documented samples compile and run, confirming a feature is actually reachable in the shipped build. DO NOT USE FOR - generating the API diff (use api-diff), confirming a managed API exists in a ref pack (use api-diff-validation), scoring features (use generate-features). +compatibility: Requires a build-metadata.json for the milestone (produced by `release-notes generate build-metadata`) and network access to ci.dot.net. Pairs with api-diff-validation, which covers the static half of the same problem. +--- + +# Validate Code Samples + +Build and run what the release notes claim. This is the **runtime verification stage** of the +pipeline, and it is the last line of defence before a component PR goes to its owner. + +[`api-verification.md`](../release-notes/references/api-verification.md) covers the *static* half of +this problem: does a managed type or member exist in the ref pack? That check is necessary and +cheap, but it is not sufficient. It cannot see JavaScript APIs, it cannot tell you what a default +value is, and it cannot tell you whether a documented sequence of calls actually works. + +## Why this stage exists + +Every one of these shipped in a draft and survived static verification. All were caught only by +running code: + +| Failure | What static verification reported | What running it showed | +| ------- | --------------------------------- | ---------------------- | +| `Blazor.pause.waitFor(fn)` | Nothing - JS surface is invisible to `dotnet-inspect` | The API does not exist. The served `blazor.web.js` exposes only `pauseCircuit` / `resumeCircuit`; the real hook is a circuit handler callback | +| `.RequireAntiforgeryToken()` | Nothing - inferred from a PR title, never compiled | Does not exist. A reflection sweep of the shipped assembly found only `DisableAntiforgery` | +| `EnableClientValidation` | Member found, so the draft looked verified | Renamed to `DisableClientValidation` **and the polarity inverted**. A name-only check actively confirmed the wrong claim | +| Documented sample page | Built cleanly | Returned HTTP 500 - the attribute silently requires a *public* property | + +The pattern: **a name is not a behavior.** Static verification answers "does this symbol exist", +which is a much weaker question than "does this documented claim hold". + +## Acquiring a build + +Do not test against whatever SDK happens to be on the machine. Test against the milestone build. + +### Preferred: derive the URL from `build-metadata.json` + +`release-notes generate build-metadata` already emits an `sdk_url` with a `{platform}` placeholder, +so no scraping is needed. Substitute the platform and download: + +```text +https://ci.dot.net/public/Sdk/{sdk_version}/dotnet-sdk-{sdk_version}-win-x64.zip +https://ci.dot.net/public/Sdk/{sdk_version}/dotnet-sdk-{sdk_version}-linux-x64.tar.gz +``` + +### Other sources + +- **** — the latest-build table in + the VMR repo. Use it when there is no `build-metadata.json` yet, or to sanity-check that the + version you derived is really the current build. It also documents the `dotnet11` NuGet feed + needed for runtime packs in self-contained scenarios. +- **** — the human-facing list of builds the team installs from. Fine for a + person, but prefer a URL derived from `build-metadata.json` for anything scripted. + +### Confirm the build matches the notes + +Each build publishes a commit manifest next to the SDK: + +```text +https://ci.dot.net/public/Sdk/{sdk_version}/productCommit-win-x64.json +``` + +```json +{ + "runtime": { "commit": "e2c1e00b...", "version": "11.0.0-preview.7.26381.103" }, + "aspnetcore": { "commit": "e2c1e00b...", "version": "11.0.0-preview.7.26381.103" }, + "sdk": { "commit": "e2c1e00b...", "version": "11.0.100-preview.7.26381.103" } +} +``` + +The `commit` is the **VMR commit** the build came from. Check it against the head ref used to +generate `changes.json`. If they disagree, you are validating a different build than the one you +documented, and any "the API is missing" conclusion is unreliable. + +### Install it scoped, not machine-wide + +Extract the archive to a scratch directory and point the environment at it. Do not install +machine-wide — a global install makes results non-reproducible and can disrupt other work on a +shared machine. + +```powershell +$root = "$env:TEMP\dotnet-p7" +Expand-Archive dotnet-sdk-*-win-x64.zip -DestinationPath $root +$env:DOTNET_ROOT = $root +$env:PATH = "$root;$env:PATH" +$env:DOTNET_MULTILEVEL_LOOKUP = "0" +dotnet --version # confirm this is the milestone build, not the machine SDK +``` + +Always print `dotnet --version` and confirm it before trusting any result. + +## What to validate + +Work through the drafted component markdown claim by claim. + +1. **Every code sample compiles.** Not "looks plausible" — actually builds against the milestone + build. A sample that does not compile is worse than no sample. +2. **Every documented default and polarity.** If the notes say a flag defaults to `true`, read the + value. Renames that invert meaning (`EnableX` becoming `DisableX`) are the highest-risk class of + change, because the name check passes while the meaning is backwards. +3. **Every JavaScript or browser-facing API.** `dotnet-inspect` cannot see these at all. Serve the + app and inspect the actual shipped script, or call the API from the page. Never document a JS API + from a PR description alone. +4. **Every documented endpoint or runtime behavior.** Request it. Record the status code. Build + success does not imply the page renders. +5. **Feature reachability.** Confirm the feature is reachable through the public surface in the + shipped build, not merely present in source. + +## Recording what you verified + +Note the build next to the claim so a reviewer can tell "this is wrong" apart from "this was checked +against a stale build": + +```markdown + +``` + +For samples that assert a specific runtime result, keep the expected result in the sample itself +(a header comment recording the expected HTTP status, for example) so drift shows up the next time +the sample is run. + +## When a claim fails validation + +Follow the escalation in +[`api-verification.md`](../release-notes/references/api-verification.md) — check the package version, +search for a rename, look for a revert, confirm the member is public. Then: + +- **Fix the notes, not the sample**, when the notes describe an API that does not exist. Rewrite the + section around what actually shipped. +- **Fix the sample, not the notes**, when the notes are right and the sample is stale. A sample + pinned to the previous preview will fail against a rename that the notes correctly documented. +- **Drop the claim** when neither holds up. A correct prose description with a PR link always beats a + confident, wrong code sample. + +## Notes + +- **Do not delegate this to a sub-agent.** Verification depends on reading real command output and + reacting to it. Summarizing agents reliably report that samples "look correct" - the failures in + the table above were all found by running the code directly. +- **A maintained samples repository is the cheapest way to run this stage.** Upgrading an existing + set of working samples to the new build surfaces renames, inverted defaults, and new analyzer + diagnostics as build errors and warnings, which is exactly the + [upgrade guidance](../release-notes/references/format-template.md) preview users need. From bdaf84621a60df94c5c12b23d5162a1cf408689d Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sun, 2 Aug 2026 19:52:25 -0700 Subject: [PATCH 3/9] Address PR feedback on the release-notes skill updates - generate-changes: replace the P7-specific coverage table with a plain list of the repos that maintain preview milestones, and add a rule that anything found via a milestone sweep must be verified independently. Milestone membership is a hand-applied label, so it shows intent, not that the change shipped. - api-verification: say "a PR associated with this milestone" rather than "a PR in this milestone", which read as GitHub milestone membership even though not every repo uses milestones per preview. - format-template: fold the preview upgrade guidance into the Breaking changes section. Preview notes already scope that section to what changed in the milestone, so it is the preview-to-preview upgrade story. Workaround removal stays called out separately since it is not a breaking change. - component-mapping, validate-code-samples: drop the historical anecdotes and keep the rules. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a851ab96-8780-407d-97d0-50eed422ead5 --- .github/skills/generate-changes/SKILL.md | 22 +++++++++++-------- .../references/api-verification.md | 4 ++-- .../references/component-mapping.md | 6 +---- .../references/format-template.md | 15 ++++++++----- .github/skills/validate-code-samples/SKILL.md | 15 ------------- 5 files changed, 26 insertions(+), 36 deletions(-) diff --git a/.github/skills/generate-changes/SKILL.md b/.github/skills/generate-changes/SKILL.md index a8b83c9b460..9b76454ba19 100644 --- a/.github/skills/generate-changes/SKILL.md +++ b/.github/skills/generate-changes/SKILL.md @@ -140,16 +140,12 @@ gh api -X GET search/issues \ --jq '.total_count' ``` -Milestone discipline varies by repo, so this check only applies where it is actually maintained. -Measured for 11.0-preview7: +Milestone discipline varies by repo, so this check only applies for the repos where it is actually maintained: -| Repo | Merged PRs in milestone | Entries in `changes.json` | Use it? | -| ---- | ----------------------- | ------------------------- | ------- | -| `dotnet/sdk` | 177 | 199 | Yes | -| `dotnet/aspnetcore` | 176 | 205 | Yes | -| `dotnet/runtime` | 583 | 692 | Yes | -| `dotnet/efcore` | 77 | 107 | Yes, with weaker coverage | -| `dotnet/roslyn`, `dotnet/razor` | no `11.x` milestones exist | 108 (roslyn) | No | +- dotnet/sdk +- dotnet/aspnetcore +- dotnet/runtime +- dotnet/efcore Rules for using it: @@ -159,6 +155,14 @@ Rules for using it: legitimately appear in `changes.json`, so the counts will not match and are not meant to. - **Only the missing direction matters.** What is worth acting on is a PR in the milestone that describes a user-facing change and has no corresponding entry in the notes. +- **Verify every addition independently before writing it up.** Milestone membership is a label + applied by hand, so it proves someone intended the PR for this release — not that the change + actually shipped in it. A PR can be milestoned and then reverted, retargeted to a later preview, + or milestoned in error. Before promoting anything found this way, confirm it appears in + `changes.json` and exists in the build (see + [`api-verification.md`](../release-notes/references/api-verification.md) and + [`validate-code-samples`](../validate-code-samples/SKILL.md)). The milestone tells you where to + look; it is not evidence on its own. - **Confirm the milestone exists before relying on its absence.** Repos without `11.x` milestones will return zero results, which means "not tracked here", not "nothing shipped". diff --git a/.github/skills/release-notes/references/api-verification.md b/.github/skills/release-notes/references/api-verification.md index 93ec505ec61..03d3dc872c3 100644 --- a/.github/skills/release-notes/references/api-verification.md +++ b/.github/skills/release-notes/references/api-verification.md @@ -125,7 +125,7 @@ A behavior observed in the Preview 7 build was traced to a fix that shipped in * real, it was verifiable, and documenting it in the Preview 7 notes would have been wrong. Anything still present from an earlier release will verify perfectly against the current build. -The rule: **every documented change must trace to a PR in this milestone**, not merely be observably +The rule: **every documented change must trace to a PR associated with this milestone**, not merely be observably true in the build. 1. **`changes.json` membership is the primary test.** If a change has no entry, it did not flow into @@ -137,7 +137,7 @@ true in the build. `changes.json` membership is the only available provenance signal. 3. **Be suspicious of anything discovered by testing rather than from `changes.json`.** Finding a nice behavior while validating samples is a good way to find *previous* releases' features. Trace - it to a PR in this milestone before promoting it. + it to a PR associated with this milestone before promoting it. This is the mirror image of the revert check below: a revert means something in `changes.json` is not in the build, and stale provenance means something in the build is not in `changes.json`. diff --git a/.github/skills/release-notes/references/component-mapping.md b/.github/skills/release-notes/references/component-mapping.md index 17e4ecf3c63..b6319b0ab1e 100644 --- a/.github/skills/release-notes/references/component-mapping.md +++ b/.github/skills/release-notes/references/component-mapping.md @@ -26,11 +26,7 @@ The agent assigns each component PR to its default assignee(s) when opening the ### Check that the owners are still current -This table is maintained by hand and goes stale as people change roles. In a single milestone, three -of the rows were wrong: one owner had retired, one had moved off the component, and one component had -been handed to someone new. - -Before opening the PRs, confirm each assignee can actually be assigned: +This table is maintained by hand and goes stale as people change roles. Before opening the PRs, confirm each assignee can actually be assigned: ```bash gh api repos/dotnet/core/assignees/ --silent # exit 0 = assignable, 404 = not diff --git a/.github/skills/release-notes/references/format-template.md b/.github/skills/release-notes/references/format-template.md index a2ff012bc6b..fc3bb74b0ca 100644 --- a/.github/skills/release-notes/references/format-template.md +++ b/.github/skills/release-notes/references/format-template.md @@ -87,16 +87,19 @@ Known component docs links: 2. **One paragraph of context** — what the feature does and why it matters in concrete terms, with PR/issue links; avoid inferred feelings or marketing-style claims 3. **Code sample** — show the feature in use 4. **Feature ordering** — highest customer impact first -5. **Breaking changes near the end** — low-score entries with `breaking_changes: true` usually belong in a short section before Bug fixes, not as full feature sections +5. **Breaking changes near the end** — low-score entries with `breaking_changes: true` usually belong in a short section before Bug fixes, not as full feature sections. In preview notes this section is also the upgrade guidance for readers coming from the previous preview — see below 6. **Preview feature callout** — when a feature is listed in `release-notes/features.json`, start its section with the standard blockquote callout from that file -7. **Call out what breaks an upgrade from the previous preview** — see below -## Upgrading from the previous preview +## Breaking changes Preview notes are read mostly by people who are already running the *previous* preview. For them the most valuable content is often not the new feature, but the thing that stops their existing code from building or working after they move to this build. +Because each preview's Breaking changes section covers what changed *in that milestone*, it is +already the preview-to-preview upgrade story. Write it for the reader doing that upgrade rather than +as an abstract list of incompatibilities with the last GA release. + Cover these when they apply. They tend to surface only when an existing project is actually upgraded to the new build, which is one of the reasons [`validate-code-samples`](../../validate-code-samples/SKILL.md) runs against a maintained sample set: @@ -106,10 +109,12 @@ to the new build, which is one of the reasons - **New analyzer diagnostics that fire on previously clean code.** Code that built without warnings on the last preview and now reports diagnostics is an upgrade issue even though nothing in the user's code changed. Name the diagnostic IDs. -- **Removed or replaced workarounds.** If a bug that required a workaround is now fixed, say so, so - users can delete the workaround rather than carrying it forward. - **Changed defaults**, where existing code keeps compiling but behaves differently. +**Removed or replaced workarounds** are the one upgrade item that is not a breaking change. If a bug +that required a workaround is now fixed, say so with the fix in Bug fixes, so users can delete the +workaround rather than carrying it forward. + Do not turn this into a changelog of everything that moved. Include an item only when a user upgrading from the previous preview would otherwise hit a build error, a new warning, or a silent behavior change. diff --git a/.github/skills/validate-code-samples/SKILL.md b/.github/skills/validate-code-samples/SKILL.md index a047ec6c92c..aa1a65cb506 100644 --- a/.github/skills/validate-code-samples/SKILL.md +++ b/.github/skills/validate-code-samples/SKILL.md @@ -14,21 +14,6 @@ this problem: does a managed type or member exist in the ref pack? That check is cheap, but it is not sufficient. It cannot see JavaScript APIs, it cannot tell you what a default value is, and it cannot tell you whether a documented sequence of calls actually works. -## Why this stage exists - -Every one of these shipped in a draft and survived static verification. All were caught only by -running code: - -| Failure | What static verification reported | What running it showed | -| ------- | --------------------------------- | ---------------------- | -| `Blazor.pause.waitFor(fn)` | Nothing - JS surface is invisible to `dotnet-inspect` | The API does not exist. The served `blazor.web.js` exposes only `pauseCircuit` / `resumeCircuit`; the real hook is a circuit handler callback | -| `.RequireAntiforgeryToken()` | Nothing - inferred from a PR title, never compiled | Does not exist. A reflection sweep of the shipped assembly found only `DisableAntiforgery` | -| `EnableClientValidation` | Member found, so the draft looked verified | Renamed to `DisableClientValidation` **and the polarity inverted**. A name-only check actively confirmed the wrong claim | -| Documented sample page | Built cleanly | Returned HTTP 500 - the attribute silently requires a *public* property | - -The pattern: **a name is not a behavior.** Static verification answers "does this symbol exist", -which is a much weaker question than "does this documented claim hold". - ## Acquiring a build Do not test against whatever SDK happens to be on the machine. Test against the milestone build. From 9439ae31c4604b8f36b8e133cf3b40af0a927f18 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sun, 2 Aug 2026 21:06:36 -0700 Subject: [PATCH 4/9] Correct how milestones get applied in the cross-check rule Milestones are applied by automation, so they are usually correct. They can be changed or applied incorrectly by hand afterwards, but that is the exception. The previous wording described them as hand-applied, which overstated the unreliability and framed verification as expecting a problem rather than confirming the expected result. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a851ab96-8780-407d-97d0-50eed422ead5 --- .github/skills/generate-changes/SKILL.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/.github/skills/generate-changes/SKILL.md b/.github/skills/generate-changes/SKILL.md index 9b76454ba19..409d34386bb 100644 --- a/.github/skills/generate-changes/SKILL.md +++ b/.github/skills/generate-changes/SKILL.md @@ -155,14 +155,14 @@ Rules for using it: legitimately appear in `changes.json`, so the counts will not match and are not meant to. - **Only the missing direction matters.** What is worth acting on is a PR in the milestone that describes a user-facing change and has no corresponding entry in the notes. -- **Verify every addition independently before writing it up.** Milestone membership is a label - applied by hand, so it proves someone intended the PR for this release — not that the change - actually shipped in it. A PR can be milestoned and then reverted, retargeted to a later preview, - or milestoned in error. Before promoting anything found this way, confirm it appears in - `changes.json` and exists in the build (see +- **Verify every addition independently before writing it up.** Milestones are applied by + automation, so they are usually right — but they can be changed or applied incorrectly by hand + afterwards, and a milestoned PR can still be reverted. Treat a milestone as reliable evidence of + where to look and weak evidence that the change shipped. Before promoting anything found this + way, confirm it appears in `changes.json` and exists in the build (see [`api-verification.md`](../release-notes/references/api-verification.md) and - [`validate-code-samples`](../validate-code-samples/SKILL.md)). The milestone tells you where to - look; it is not evidence on its own. + [`validate-code-samples`](../validate-code-samples/SKILL.md)). The expected outcome is that it + checks out; the point is to catch the occasional one that does not. - **Confirm the milestone exists before relying on its absence.** Repos without `11.x` milestones will return zero results, which means "not tracked here", not "nothing shipped". From a0acdceea14413ea24a0b46cf8acad51691440ec Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Sun, 2 Aug 2026 21:10:55 -0700 Subject: [PATCH 5/9] Broaden the gh pr edit workaround to the whole command The Projects (classic) GraphQL failure is not specific to the assignee flags - gh pr edit --body-file fails against dotnet/core the same way and silently leaves the body unchanged. Note the REST PATCH for title and body edits too. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a851ab96-8780-407d-97d0-50eed422ead5 --- .github/skills/release-notes/references/pr-layout.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/.github/skills/release-notes/references/pr-layout.md b/.github/skills/release-notes/references/pr-layout.md index 39c355f7e5e..23b4f611a7c 100644 --- a/.github/skills/release-notes/references/pr-layout.md +++ b/.github/skills/release-notes/references/pr-layout.md @@ -30,14 +30,20 @@ Order matters, and two of these steps fail silently. the repo: the API returns success and the PR is created with the assignee missing. Re-read the PR and compare against the intended list rather than trusting the exit code. -### gh pr edit does not work for assignees on this repo +### gh pr edit does not work on this repo -`gh pr edit --add-assignee` / `--remove-assignee` fails against `dotnet/core` with a Projects -(classic) GraphQL deprecation error, leaving assignees unchanged. Use the REST endpoints: +`gh pr edit` fails against `dotnet/core` with a Projects (classic) GraphQL deprecation error and +leaves the PR unchanged. This affects the whole command, not just one flag — `--add-assignee`, +`--remove-assignee`, and `--body` / `--body-file` all fail the same way. It exits non-zero, but the +error text is about Projects rather than about what you were trying to change, so it is easy to +mistake for a warning. Use the REST endpoints: ```bash gh api -X POST repos/dotnet/core/issues/{number}/assignees -f "assignees[]=" gh api -X DELETE repos/dotnet/core/issues/{number}/assignees -f "assignees[]=" + +# Editing the PR title or body +'{"body": "..."}' | gh api -X PATCH repos/dotnet/core/pulls/{number} --input - ``` ## Merge flow From 412e38824f16a7ff3949310d76e147fa5d922d64 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Mon, 3 Aug 2026 08:12:32 -0700 Subject: [PATCH 6/9] Make the bug fixes section a flat list of PR links Each bug fix is one bullet: a single markdown link pointing at the PR, with a cleaned-up version of the PR or issue title as the display text. No area sub-grouping, no bundling several PRs into one bullet, and no trailing org/repo #number citation, since the link already carries it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a851ab96-8780-407d-97d0-50eed422ead5 --- .../references/editorial-rules.md | 28 +++++++++++++++---- .../references/format-template.md | 2 +- 2 files changed, 23 insertions(+), 7 deletions(-) diff --git a/.github/skills/release-notes/references/editorial-rules.md b/.github/skills/release-notes/references/editorial-rules.md index d4cf65d3c2e..4223de70b96 100644 --- a/.github/skills/release-notes/references/editorial-rules.md +++ b/.github/skills/release-notes/references/editorial-rules.md @@ -108,18 +108,34 @@ Thank you contributors! ❤️ ## Bug fixes section -After features but before community contributors, include a grouped bug fix summary when there are noteworthy fixes. When citing the source work, use linked `org/repo #number` references with a space before `#`: +After features but before community contributors, include a bug fix summary when there are noteworthy fixes. + +Keep it a **flat bulleted list with one bullet per fix**. Each bullet is a single markdown link whose target is the PR that made the fix and whose display text is a cleaned-up version of the PR or issue title: ```markdown ## Bug fixes -- **System.Net.Http** - - Fixed authenticated proxy credential handling ([dotnet/runtime #123363](https://github.com/dotnet/runtime/issues/123363)) -- **System.Collections** - - Fixed integer overflow in ImmutableArray range validation ([dotnet/runtime #124042](https://github.com/dotnet/runtime/pull/124042)) +- [Fix authenticated proxy credential handling](https://github.com/dotnet/runtime/pull/123363) +- [Fix integer overflow in ImmutableArray range validation](https://github.com/dotnet/runtime/pull/124042) ``` -Group by namespace/area. Don't include test-only, CI, or infra fixes. +Rules: + +- **One fix per bullet.** If a single behavior was fixed by several PRs, give each PR its own bullet rather than bundling links into one sentence. +- **Link to the PR**, not the issue, even when the issue has the better title. Take the wording from whichever reads more clearly and point the link at the PR. +- **No sub-grouping and no category headings.** Do not nest bullets under area names. +- **No trailing prose.** The bullet is the link and nothing else — no explanation after it, and no `org/repo #number` citation, since the link already carries that. + +This overrides the general `org/repo #number` citation style used elsewhere in the notes; that style still applies in feature sections and breaking changes. + +Cleaning up the title means making it read as a plain description of the fix: + +- Drop branch and process prefixes (`[release/11.0]`, `[main]`, backport markers). +- Drop trailing issue references (`(#12345)`, `Fixes #123`). +- Expand cryptic shorthand into words a reader outside the team would recognize. +- Keep it a single line. If the title is unintelligible without context, rewrite it as a short description of the fix rather than pasting the raw title. + +Don't include test-only, CI, or infra fixes. ## Preview-to-preview feedback fixes diff --git a/.github/skills/release-notes/references/format-template.md b/.github/skills/release-notes/references/format-template.md index fc3bb74b0ca..2638eb93d70 100644 --- a/.github/skills/release-notes/references/format-template.md +++ b/.github/skills/release-notes/references/format-template.md @@ -24,7 +24,7 @@ Standard document structure for .NET release notes markdown files. ## Bug fixes -- **Category** — Fix description +- [Cleaned up PR or issue title](https://github.com///pull/NNNNN) ## Community contributors From cb807eefe0335bd22e9cfc6317343e6032efef4e Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Mon, 3 Aug 2026 08:57:17 -0700 Subject: [PATCH 7/9] Keep the area grouping in the bug fixes section One bullet per PR with a cleaned-up title is easier to scan when the fixes are still grouped by namespace or area, so keep the grouping one level deep rather than a single flat list. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a851ab96-8780-407d-97d0-50eed422ead5 --- .../skills/release-notes/references/editorial-rules.md | 10 ++++++---- .../skills/release-notes/references/format-template.md | 3 ++- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/.github/skills/release-notes/references/editorial-rules.md b/.github/skills/release-notes/references/editorial-rules.md index 4223de70b96..89a0f28c7b7 100644 --- a/.github/skills/release-notes/references/editorial-rules.md +++ b/.github/skills/release-notes/references/editorial-rules.md @@ -110,20 +110,22 @@ Thank you contributors! ❤️ After features but before community contributors, include a bug fix summary when there are noteworthy fixes. -Keep it a **flat bulleted list with one bullet per fix**. Each bullet is a single markdown link whose target is the PR that made the fix and whose display text is a cleaned-up version of the PR or issue title: +Group the fixes by namespace or area, with a **flat list of one bullet per fix** under each group. Each bullet is a single markdown link whose target is the PR that made the fix and whose display text is a cleaned-up version of the PR or issue title: ```markdown ## Bug fixes -- [Fix authenticated proxy credential handling](https://github.com/dotnet/runtime/pull/123363) -- [Fix integer overflow in ImmutableArray range validation](https://github.com/dotnet/runtime/pull/124042) +- **System.Net.Http** + - [Fix authenticated proxy credential handling](https://github.com/dotnet/runtime/pull/123363) +- **System.Collections** + - [Fix integer overflow in ImmutableArray range validation](https://github.com/dotnet/runtime/pull/124042) ``` Rules: - **One fix per bullet.** If a single behavior was fixed by several PRs, give each PR its own bullet rather than bundling links into one sentence. - **Link to the PR**, not the issue, even when the issue has the better title. Take the wording from whichever reads more clearly and point the link at the PR. -- **No sub-grouping and no category headings.** Do not nest bullets under area names. +- **Group by namespace or area**, exactly one level deep. Use the namespace where the component has one (`System.Text.Json`), otherwise a feature area (`Blazor`, `JIT / code generation`). - **No trailing prose.** The bullet is the link and nothing else — no explanation after it, and no `org/repo #number` citation, since the link already carries that. This overrides the general `org/repo #number` citation style used elsewhere in the notes; that style still applies in feature sections and breaking changes. diff --git a/.github/skills/release-notes/references/format-template.md b/.github/skills/release-notes/references/format-template.md index 2638eb93d70..8a9d58ddeb0 100644 --- a/.github/skills/release-notes/references/format-template.md +++ b/.github/skills/release-notes/references/format-template.md @@ -24,7 +24,8 @@ Standard document structure for .NET release notes markdown files. ## Bug fixes -- [Cleaned up PR or issue title](https://github.com///pull/NNNNN) +- **Namespace or area** + - [Cleaned up PR or issue title](https://github.com///pull/NNNNN) ## Community contributors From 280ef231166bc17eafa134db0c47097617e6fa2b Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Tue, 4 Aug 2026 08:31:44 -0700 Subject: [PATCH 8/9] Clarify milestone evidence strength Milestones are applied by automation and are usually correct, so describe them as strong but not conclusive evidence that a change shipped. Keep the separate changes.json and build verification step for the occasional incorrect or stale milestone assignment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a851ab96-8780-407d-97d0-50eed422ead5 --- .github/skills/generate-changes/SKILL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/skills/generate-changes/SKILL.md b/.github/skills/generate-changes/SKILL.md index 409d34386bb..7c1def69d30 100644 --- a/.github/skills/generate-changes/SKILL.md +++ b/.github/skills/generate-changes/SKILL.md @@ -158,8 +158,8 @@ Rules for using it: - **Verify every addition independently before writing it up.** Milestones are applied by automation, so they are usually right — but they can be changed or applied incorrectly by hand afterwards, and a milestoned PR can still be reverted. Treat a milestone as reliable evidence of - where to look and weak evidence that the change shipped. Before promoting anything found this - way, confirm it appears in `changes.json` and exists in the build (see + where to look and strong but not conclusive evidence that the change shipped. Before promoting + anything found this way, confirm it appears in `changes.json` and exists in the build (see [`api-verification.md`](../release-notes/references/api-verification.md) and [`validate-code-samples`](../validate-code-samples/SKILL.md)). The expected outcome is that it checks out; the point is to catch the occasional one that does not. From 0753d6de2ca32faab2a13f35be564adea936b6f5 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Fri, 7 Aug 2026 13:44:01 -0700 Subject: [PATCH 9/9] Use SDK build listings for sample validation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a851ab96-8780-407d-97d0-50eed422ead5 --- .github/skills/validate-code-samples/SKILL.md | 34 ++++++++----------- 1 file changed, 14 insertions(+), 20 deletions(-) diff --git a/.github/skills/validate-code-samples/SKILL.md b/.github/skills/validate-code-samples/SKILL.md index aa1a65cb506..fc91a39ee4f 100644 --- a/.github/skills/validate-code-samples/SKILL.md +++ b/.github/skills/validate-code-samples/SKILL.md @@ -1,7 +1,7 @@ --- name: validate-code-samples -description: Verify release notes claims by building and running them against the actual .NET build for the milestone. Covers acquiring a scoped SDK from ci.dot.net, exercising every documented API and code sample, and catching the errors that static API verification cannot see - non-existent JavaScript APIs, inverted defaults, and runtime failures. USE FOR - validating a drafted component's release notes before the PR goes up, checking that documented samples compile and run, confirming a feature is actually reachable in the shipped build. DO NOT USE FOR - generating the API diff (use api-diff), confirming a managed API exists in a ref pack (use api-diff-validation), scoring features (use generate-features). -compatibility: Requires a build-metadata.json for the milestone (produced by `release-notes generate build-metadata`) and network access to ci.dot.net. Pairs with api-diff-validation, which covers the static half of the same problem. +description: Verify release notes claims by building and running them against the actual .NET build for the milestone. Covers acquiring a scoped SDK from the latest builds table linked from the dotnet/sdk repository, exercising every documented API and code sample, and catching the errors that static API verification cannot see - non-existent JavaScript APIs, inverted defaults, and runtime failures. USE FOR - validating a drafted component's release notes before the PR goes up, checking that documented samples compile and run, confirming a feature is actually reachable in the shipped build. DO NOT USE FOR - generating the API diff (use api-diff), confirming a managed API exists in a ref pack (use api-diff-validation), scoring features (use generate-features). +compatibility: Requires network access to GitHub and the public .NET build artifacts. Uses build-metadata.json for the milestone when available to confirm build provenance. Pairs with api-diff-validation, which covers the static half of the same problem. --- # Validate Code Samples @@ -16,26 +16,19 @@ value is, and it cannot tell you whether a documented sequence of calls actually ## Acquiring a build -Do not test against whatever SDK happens to be on the machine. Test against the milestone build. +Do not test against whatever SDK happens to be on the machine. Select an appropriate build for the +milestone from the build listings linked by the .NET SDK repository. -### Preferred: derive the URL from `build-metadata.json` +### Select a build from the .NET SDK repository -`release-notes generate build-metadata` already emits an `sdk_url` with a `{platform}` placeholder, -so no scraping is needed. Substitute the platform and download: +Start from the [`dotnet/sdk` Installing the SDK +section](https://github.com/dotnet/sdk#installing-the-sdk) and follow its **.NET SDK latest builds +table** link. Select the column that matches the milestone's SDK feature band or release branch, +then download the archive for the validation machine's platform. Preview notes should use the +matching preview column, not the build from `main`. -```text -https://ci.dot.net/public/Sdk/{sdk_version}/dotnet-sdk-{sdk_version}-win-x64.zip -https://ci.dot.net/public/Sdk/{sdk_version}/dotnet-sdk-{sdk_version}-linux-x64.tar.gz -``` - -### Other sources - -- **** — the latest-build table in - the VMR repo. Use it when there is no `build-metadata.json` yet, or to sanity-check that the - version you derived is really the current build. It also documents the `dotnet11` NuGet feed - needed for runtime packs in self-contained scenarios. -- **** — the human-facing list of builds the team installs from. Fine for a - person, but prefer a URL derived from `build-metadata.json` for anything scripted. +The builds table also documents the public NuGet feed needed when development builds must acquire +runtime packs or other assets that aren't included in the SDK archive. ### Confirm the build matches the notes @@ -55,7 +48,8 @@ https://ci.dot.net/public/Sdk/{sdk_version}/productCommit-win-x64.json The `commit` is the **VMR commit** the build came from. Check it against the head ref used to generate `changes.json`. If they disagree, you are validating a different build than the one you -documented, and any "the API is missing" conclusion is unreliable. +documented, and any "the API is missing" conclusion is unreliable. When `build-metadata.json` is +available, also compare its SDK version and VMR ref with the selected build before testing. ### Install it scoped, not machine-wide