OPS-157454: Upload test coverage to Codacy from CI - #26
Conversation
Adds a coverage job that runs the test project as a Microsoft.Testing.Platform executable with Microsoft.Testing.Extensions.CodeCoverage and uploads the Cobertura report to Codacy, on the self-hosted pdl-public runners. Excluded tests: none - every test is offline (20 tests). Local line coverage (clean Linux container, no user secrets): 45.9% Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Coverage variation | Report missing for b411da91 |
| Diff coverage | ✅ ∅ diff coverage |
Coverage variation details
Coverable lines Covered lines Coverage Common ancestor commit (b411da9) Report Missing Report Missing Report Missing Head commit (34ceb0c) 790 363 45.95% Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch:
<coverage of head commit> - <coverage of common ancestor commit>
Diff coverage details
Coverable lines Covered lines Diff coverage Pull request (#26) 0 0 ∅ (not applicable) Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified:
<covered lines added or modified>/<coverable lines added or modified> * 100%
1 Codacy didn't receive coverage data for the commit, or there was an error processing the received data. Check your integration for errors and validate that your coverage setup is correct.
AI Reviewer: first review requested successfully. AI can make mistakes. Always validate suggestions.
TIP This summary will be updated as you push new changes.
There was a problem hiding this comment.
Pull Request Overview
While this PR correctly establishes the structure for CI coverage reporting, there is a fundamental mismatch in the file paths used to locate the coverage report after generation. Codacy analysis currently reports 'MissingRequirements' for coverage, confirming that the generated Cobertura file is not being found by the upload step.
Additionally, the acceptance criterion regarding exclusion rules is partially unmet: the regex for excluding 'obj' directories uses Windows-specific separators which will fail on the Linux runner. The setup of the .NET environment also needs to be adjusted to properly utilize inputs for self-hosted runner compatibility. These issues should be resolved before merging to ensure the coverage reporting pipeline functions as intended.
About this PR
- The workflow contains a path discrepancy between the test execution step and the subsequent upload steps. While the test runner is executed from the root, the upload steps attempt to locate the report deep within the build output directory. This prevents the primary goal of the PR (coverage reporting) from succeeding.
Test suggestions
- Execution of tests with coverage collection enabled
- Application of exclusion filters (generated code, obj folders) during coverage collection
- Upload of coverage results to Codacy with failure tolerance
- Persistence of coverage reports as workflow artifacts
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: coverage | ||
| path: SchemaMagic.Tests/bin/Debug/net10.0/TestResults/coverage.cobertura.xml |
There was a problem hiding this comment.
🔴 HIGH RISK
Update the artifact path to match the actual output location, likely 'TestResults/coverage.cobertura.xml', to ensure the report is correctly archived.
| uses: codacy/codacy-coverage-reporter-action@v1 | ||
| with: | ||
| project-token: ${{ secrets.CODACY_PROJECT_TOKEN }} | ||
| coverage-reports: SchemaMagic.Tests/bin/Debug/net10.0/TestResults/coverage.cobertura.xml |
There was a problem hiding this comment.
🔴 HIGH RISK
The path to the coverage report likely doesn't match the output of the run command. When running the executable from the root with '--coverage-output coverage.cobertura.xml', the Microsoft.Testing.Platform typically places results in a 'TestResults' folder in the current working directory. Update this path to 'TestResults/coverage.cobertura.xml' to resolve the 'MissingRequirements' error reported by Codacy.
| </Attributes> | ||
| <Sources> | ||
| <Exclude> | ||
| <Source>.*\\obj\\.*</Source> |
There was a problem hiding this comment.
🟡 MEDIUM RISK
The regex for excluding the 'obj' directory uses literal backslashes (\\), which are Windows-specific. Since this job runs on a Linux runner (pdl-public), the regex will fail to match. Use a platform-agnostic pattern: <Source>.*/obj/.*</Source>.
| env: | ||
| # The runner user cannot write /usr/share/dotnet, the default install location. | ||
| DOTNET_INSTALL_DIR: ${{ runner.temp }}/dotnet | ||
| with: | ||
| dotnet-version: 10.0.x | ||
|
|
There was a problem hiding this comment.
🟡 MEDIUM RISK
Suggestion: Move the installation directory configuration from the 'env' block to the 'with' block using the 'dotnet-install-dir' parameter to ensure the action installs .NET to the correct location on self-hosted runners.
Adds a
coveragejob to CI that collects test coverage and uploads it to Codacy.UseMicrosoftTestingPlatformRunner) withMicrosoft.Testing.Extensions.CodeCoverage, writing a Cobertura report.coverage.configports the generated-code exclusions used across the estate, so figures stay comparable.codacy/codacy-coverage-reporter-actionusing theCODACY_PROJECT_TOKENsecret (continue-on-error, so a Codacy outage cannot turn a passing run red), and kept as a 7-day artifact.pdl-publicrunners, not a GitHub-hosted runner.Excluded tests: none - every test is offline (20 tests).
CI runs only tests that need no credentials. Tests are selected by what passes in a clean Linux container with no user secrets, exactly as CI sees it.
Local line coverage: 45.9%
Jira: https://jira.panoramicdata.com/browse/OPS-157454
🤖 Generated with Claude Code