Skip to content

Microsoft.Testing.Platform support in the Test Explorer - #250

Open
validide wants to merge 1 commit into
Nimblesite:mainfrom
validide:mtp-test-explorer
Open

validide wants to merge 1 commit into
Nimblesite:mainfrom
validide:mtp-test-explorer

Conversation

@validide

@validide validide commented Sep 9, 2026

Copy link
Copy Markdown

Closes #249. The Test Explorer was VSTest end to end, so a Microsoft.Testing.Platform (MTP) project showed an empty Testing view. On the .NET 10 SDK, MTP v2 removed the VSTest shim and xunit.v3 4.0.0 uses MTP v2 by default, so that is now the ordinary case: --nologo is rejected in MTP mode (exit 5, zero tests), dotnet vstest cannot load an MTP-only module at all, and neither --filter nor --logger trx exists there.

Product changes

  • feat(vscode): discover and run MTP projects. A second path beside VSTest that shares everything after the TRX file. The runner is chosen per target: the global.json test.runner opt-in selects MTP outright and skips the two doomed VSTest passes; without one, the VSTest passes run first and only a sweep that produced no fully-qualified name asks MSBuild for IsTestingPlatformApplication (IsTestProject MUST NOT be used — xunit.v3 leaves it empty). A VSTest solution therefore pays nothing ([TEST-MTP-DETECT]).
  • feat(vscode): ask the test MODULE, not dotnet test. MTP prints no Test run for … banner, so the modules come from dotnet sln list plus -getProperty:TargetPath — the only source that survives a custom AssemblyName, OutputPath, ArtifactsPath or RuntimeIdentifier. Each module then answers dotnet exec <module> --list-tests json directly; dotnet test does not forward the json argument (Allow dotnet test --list-tests json dotnet/sdk#49754) ([TEST-MTP-MODULES], [TEST-MTP-DISCOVERY]).
  • feat(vscode): the id comes from the listing's type block. namespace + typeName + methodName reproduces the [TEST-DISCOVERY-FQN] table exactly — F# backtick names carrying SPACES, the F# [<TestClass>] nested-type +, and no row data, so a data-driven test's rows collapse onto the one id they share. It must NOT come from displayName: MSTest renders that as the BARE method name, which is issue Test Explorer: MSTest tests not discovered (dotnet test --list-tests emits bare DisplayName, not FQN) #180 in its MTP shape. The reconstructed id equals the className + . + name pair the TRX report holds, so [TEST-RUN-TRX] attributes MTP outcomes with no change at all. The listing also carries a source location, so MTP rows get a file and line the VSTest path never had.
  • feat(vscode): run by --filter-uid, report by --report-trx. One invocation per module, uids batched under the Windows command-line ceiling. The uids are LITERAL, so the [TEST-FILTER-ESCAPE] grammar does not apply and must not be used — an NUnit uid is Ns.Class.Adds_Case(2,2,4), and escaping its parentheses would match nothing ([TEST-MTP-RUN]).
  • fix(vscode): recover when a bridged framework REFUSES the selection. NUnit translates --filter-uid back into a VSTest filter expression and then rejects its own translation for any uid carrying a SPACE — every idiomatic F# backtick binding. The whole module reported nothing and four runnable tests showed as phantom failures. That module is now re-run ONCE unfiltered and the outcomes read back by name, the rule [TEST-FILTER-ESCAPE] already sets for VSTest. A rejected OPTION earns no retry: it would be rejected again.
  • fix(vscode): name the package a module is missing. --report-trx is an extension, not part of MTP; a module without Microsoft.Testing.Extensions.TrxReport exits 5. Silence would report every selected test as "No result reported" and hide the cause.
  • fix(vscode): find a waiting host behind a prefix. An MTP module IS the test host and prints Waiting for debugger to attach... Process Id: … — the same text as VSTest behind a prefix. Anchored to the start of the line, every MTP debug run hung on a module nothing ever attached to ([TEST-MTP-DEBUG]).
  • fix(vscode): read Cobertura at both depths. MTP collects with --coverage --coverage-output-format cobertura and writes <guid>.cobertura.xml directly into the results directory, where coverlet.collector writes one level down ([TEST-COVERAGE]).

Tests

New testexplorer-mtp chunk, its own job because it builds a SECOND six-project fixture solution — xunit.v3, MSTest and NUnit, each in C# and F#, F# first. Together with testexplorer-frameworks it would build twelve test projects in one job and blow the 15-minute ceiling ([DIST-CI-WIN-VSIX]).

  • test-explorer-mtp.test.ts — the opt-in and the MSBuild probe, module resolution, the tree for all six fixtures, the MSTest bare display name that must never become an id, and the source location that NUnit does not report and must not be invented.
  • test-explorer-mtp-outcomes.test.ts — pass/fail/skip across all six projects with each framework's OWN assertion text, a theory whose rows disagree, the class row, the NUnit refusal recovering and the C# selection that must NOT be retried, and the missing-TrxReport message.
  • test-explorer-mtp-parsers.test.ts — the readers at their boundary: the opt-in against every decoy that merely mentions MTP, a BOM, an unknown schemaVersion, a missing type, the uid batcher, and the pid line in both its bare and prefixed forms.

The shared assertFailed hardcoded xUnit's Assert.Equal() Failure, which no MSTest or NUnit failure carries — it had only ever been fed xUnit fixtures. It now takes the framework's own text, defaulting to xUnit's, so every existing caller is unchanged.

Refactoring

testing.ts was 628 lines against a 500 ceiling; it is 480, with the tree builder, the run profiles, the result cache and the dotnet queue in their own modules. Deslop found isRecord written out identically five times — it now lives in utils.ts, re-exported from dap-emulate so the ten DAP importers are untouched — and one command-line batcher now serves the assembly list, the VSTest filter and the MTP uids. Duplication measures 15.69% against the 20.0% ceiling.

🤖 Generated with Claude Code

TLDR

Details

How Do The Automated Tests Prove It Works?

Closes Nimblesite#249. The Test Explorer was VSTest end to end, so a
Microsoft.Testing.Platform (MTP) project showed an empty Testing view.
On the .NET 10 SDK, MTP v2 removed the VSTest shim and `xunit.v3` 4.0.0
uses MTP v2 by default, so that is now the ordinary case: `--nologo` is
rejected in MTP mode (exit 5, zero tests), `dotnet vstest` cannot load
an MTP-only module at all, and neither `--filter` nor `--logger trx`
exists there.

## Product changes

* **`feat(vscode)`: discover and run MTP projects.** A second path
beside VSTest that shares everything after the TRX file. The runner is
chosen per target: the `global.json` `test.runner` opt-in selects MTP
outright and skips the two doomed VSTest passes; without one, the
VSTest passes run first and only a sweep that produced no
fully-qualified name asks MSBuild for `IsTestingPlatformApplication`
(`IsTestProject` MUST NOT be used — `xunit.v3` leaves it empty). A
VSTest solution therefore pays nothing ([TEST-MTP-DETECT]).
* **`feat(vscode)`: ask the test MODULE, not `dotnet test`.** MTP prints
no `Test run for …` banner, so the modules come from `dotnet sln list`
plus `-getProperty:TargetPath` — the only source that survives a custom
`AssemblyName`, `OutputPath`, `ArtifactsPath` or `RuntimeIdentifier`.
Each module then answers `dotnet exec <module> --list-tests json`
directly; `dotnet test` does not forward the `json` argument
(dotnet/sdk#49754) ([TEST-MTP-MODULES], [TEST-MTP-DISCOVERY]).
* **`feat(vscode)`: the id comes from the listing's `type` block.**
`namespace` + `typeName` + `methodName` reproduces the
[TEST-DISCOVERY-FQN] table exactly — F# backtick names carrying SPACES,
the F# `[<TestClass>]` nested-type `+`, and no row data, so a
data-driven test's rows collapse onto the one id they share. It must
NOT come from `displayName`: MSTest renders that as the BARE method
name, which is issue Nimblesite#180 in its MTP shape. The reconstructed id equals
the `className` + `.` + `name` pair the TRX report holds, so
[TEST-RUN-TRX] attributes MTP outcomes with no change at all. The
listing also carries a source location, so MTP rows get a file and line
the VSTest path never had.
* **`feat(vscode)`: run by `--filter-uid`, report by `--report-trx`.**
One invocation per module, uids batched under the Windows
command-line ceiling. The uids are LITERAL, so the
[TEST-FILTER-ESCAPE] grammar does not apply and must not be used — an
NUnit uid is `Ns.Class.Adds_Case(2,2,4)`, and escaping its parentheses
would match nothing ([TEST-MTP-RUN]).
* **`fix(vscode)`: recover when a bridged framework REFUSES the
selection.** NUnit translates `--filter-uid` back into a VSTest filter
expression and then rejects its own translation for any uid carrying a
SPACE — every idiomatic F# backtick binding. The whole module reported
nothing and four runnable tests showed as phantom failures. That module
is now re-run ONCE unfiltered and the outcomes read back by name, the
rule [TEST-FILTER-ESCAPE] already sets for VSTest. A rejected OPTION
earns no retry: it would be rejected again.
* **`fix(vscode)`: name the package a module is missing.**
`--report-trx` is an extension, not part of MTP; a module without
`Microsoft.Testing.Extensions.TrxReport` exits 5. Silence would report
every selected test as "No result reported" and hide the cause.
* **`fix(vscode)`: find a waiting host behind a prefix.** An MTP module
IS the test host and prints `Waiting for debugger to attach... Process
Id: …` — the same text as VSTest behind a prefix. Anchored to the start
of the line, every MTP debug run hung on a module nothing ever attached
to ([TEST-MTP-DEBUG]).
* **`fix(vscode)`: read Cobertura at both depths.** MTP collects with
`--coverage --coverage-output-format cobertura` and writes
`<guid>.cobertura.xml` directly into the results directory, where
`coverlet.collector` writes one level down ([TEST-COVERAGE]).

## Tests

New `testexplorer-mtp` chunk, its own job because it builds a SECOND
six-project fixture solution — `xunit.v3`, MSTest and NUnit, each in C#
and F#, F# first. Together with `testexplorer-frameworks` it would
build twelve test projects in one job and blow the 15-minute ceiling
([DIST-CI-WIN-VSIX]).

* **`test-explorer-mtp.test.ts`** — the opt-in and the MSBuild probe,
module resolution, the tree for all six fixtures, the MSTest bare
display name that must never become an id, and the source location that
NUnit does not report and must not be invented.
* **`test-explorer-mtp-outcomes.test.ts`** — pass/fail/skip across all
six projects with each framework's OWN assertion text, a theory whose
rows disagree, the class row, the NUnit refusal recovering and the C#
selection that must NOT be retried, and the missing-TrxReport message.
* **`test-explorer-mtp-parsers.test.ts`** — the readers at their
boundary: the opt-in against every decoy that merely mentions MTP, a
BOM, an unknown `schemaVersion`, a missing `type`, the uid batcher, and
the pid line in both its bare and prefixed forms.

The shared `assertFailed` hardcoded xUnit's `Assert.Equal() Failure`,
which no MSTest or NUnit failure carries — it had only ever been fed
xUnit fixtures. It now takes the framework's own text, defaulting to
xUnit's, so every existing caller is unchanged.

## Refactoring

`testing.ts` was 628 lines against a 500 ceiling; it is 480, with the
tree builder, the run profiles, the result cache and the `dotnet` queue
in their own modules. Deslop found `isRecord` written out identically
five times — it now lives in `utils.ts`, re-exported from `dap-emulate`
so the ten DAP importers are untouched — and one command-line batcher
now serves the assembly list, the VSTest filter and the MTP uids.
Duplication measures 15.69% against the 20.0% ceiling.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Microsoft.Testing.Platform (MTP) support in Test Explorer

1 participant