Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -1023,6 +1023,16 @@ lint-cjs: fmt-check-cjs validate-cjs-syntax check-node-version
lint-json: fmt-check-json
@echo "✓ JSON formatting validated"

# Full-repository error-message audit (non-blocking report).
# Reports pre-existing error-message violations across the repo so the debt can
# be tracked as a metric over time. Always exits 0.
.PHONY: lint-error-messages-report
lint-error-messages-report:
@echo "Building custom linters..."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/grill-with-docs] The binary is built to /tmp/gh-aw-linters, which is the same path used by the existing golint-custom target. If both targets run in the same CI job, the second build overwrites the binary and could cause non-deterministic results.

💡 Suggested fix

Use a distinct output path for this target:

lint-error-messages-report:
	env -u GOOS -u GOARCH go build -o /tmp/gh-aw-linters-report ./cmd/linters
	/tmp/gh-aw-linters-report -errormessage -errormessage.full-repo $(LINTER_PACKAGES) || true

@copilot please address this.

@env -u GOOS -u GOARCH go build -o /tmp/gh-aw-linters ./cmd/linters
@echo "Auditing error messages across $(LINTER_PACKAGES) (non-blocking)..."
@/tmp/gh-aw-linters -errormessage -errormessage.full-repo $(LINTER_PACKAGES) || true

# Lint error messages for quality compliance
.PHONY: lint-errors
lint-errors:
Expand Down Expand Up @@ -1399,6 +1409,7 @@ help:
@echo " validate-cjs-syntax - Syntax-check all non-test .cjs files (catches module-load SyntaxErrors)"
@echo " lint-json - Lint JSON files in pkg directory (excluding actions/setup/js)"
@echo " lint-errors - Lint error messages for quality compliance"
@echo " lint-error-messages-report - Non-blocking full-repo error message audit"
@echo " validate-otel-contract - Validate the gh-aw OpenTelemetry compatibility contract"
@echo " lint-action-sh - Lint action shell scripts for python/python3 invocations"
@echo " shellcheck-setup-sh - Run shellcheck on actions/setup/sh scripts"
Expand Down
4 changes: 2 additions & 2 deletions pkg/linters/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ This package currently provides custom Go analyzers in the following subpackages
- `deferinloop` — reports `defer` statements placed directly inside `for`/`range` loop bodies, which execute when the enclosing function returns rather than each iteration and can cause resource leaks.
- `errorfwrapv` — reports `fmt.Errorf` calls that pass error arguments without `%w` wrapping.
- `excessivefuncparams` — reports function declarations that exceed a configurable parameter-count threshold.
- `errormessage` — reports non-actionable error-message patterns in changed files.
- `errormessage` — reports non-actionable error-message patterns in changed files; pass `-errormessage.full-repo` (or `-errormessage.changed-files=all`) to audit the whole repository.
- `errortypeassertion` — reports type assertions from `error` to concrete types and recommends `errors.As`.
- `errstringmatch` — reports `strings.Contains(err.Error(), "...")` patterns and recommends `errors.Is` / `errors.As`.
- `fileclosenotdeferred` — reports non-deferred file `Close()` calls that can leak resources.
Expand Down Expand Up @@ -85,7 +85,7 @@ This package currently provides custom Go analyzers in the following subpackages
| `deferinloop` | Custom `go/analysis` analyzer that flags `defer` statements inside `for`/`range` loop bodies that execute when the enclosing function returns rather than each iteration |
| `errorfwrapv` | Custom `go/analysis` analyzer that flags `fmt.Errorf` calls that pass error arguments without `%w` wrapping |
| `excessivefuncparams` | Custom `go/analysis` analyzer that flags function declarations with too many positional parameters |
| `errormessage` | Custom `go/analysis` analyzer that flags non-actionable error message patterns in changed files |
| `errormessage` | Custom `go/analysis` analyzer that flags non-actionable error message patterns in changed files (or all files with `-errormessage.full-repo`) |
| `errortypeassertion` | Custom `go/analysis` analyzer that flags type assertions from `error` to concrete types and recommends `errors.As` |
| `errstringmatch` | Custom `go/analysis` analyzer that flags brittle `strings.Contains(err.Error(), "...")` checks |
| `execcommandwithoutcontext` | Custom `go/analysis` analyzer that flags `exec.Command(...)` calls that should use `exec.CommandContext(...)` in context-receiving functions |
Expand Down
34 changes: 32 additions & 2 deletions pkg/linters/errormessage/errormessage.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,23 +24,42 @@ var (
// changedFilesCSV allows CI to scope linting to changed files only,
// preventing legacy violations from blocking incremental adoption.
changedFilesCSV string
// fullRepo enables auditing every analyzed file instead of only the
// changed ones, so pre-existing violations can be tracked as a metric.
fullRepo bool
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/codebase-design] The fullRepo package-level variable is a global flag that persists across test runs in the same process. Setting it via Analyzer.Flags.Set in one test will bleed into subsequent tests unless explicitly reset, which the existing t.Cleanup handles — but only if the test passes. A panic or t.Fatal before cleanup registers won't reset the flag.

💡 Suggested improvement

This is an inherent limitation of the go/analysis framework's global-flag model, but it is worth documenting in a comment so future contributors understand why tests must always call t.Cleanup to reset flags:

// NOTE: fullRepo and changedFilesCSV are global package variables bound to
// Analyzer.Flags. Tests that mutate them must reset them via t.Cleanup to
// prevent cross-test pollution.
var (
    changedFilesCSV string
    fullRepo        bool
)

Also consider registering the cleanup before any operation that could fail so the reset always runs.

@copilot please address this.

// fullRepoSentinel is the -changed-files value that enables full-repository
// auditing, equivalent to passing -full-repo.
const fullRepoSentinel = "all"

// Analyzer is the errormessage analysis pass.
var Analyzer = analyzerutil.New("errormessage", "reports non-actionable error message patterns in changed files", run)
var Analyzer = analyzerutil.New("errormessage", "reports non-actionable error message patterns in changed files (or all files with -full-repo)", run)

func init() {
Analyzer.Flags.StringVar(&changedFilesCSV, "changed-files", "", "comma-separated list of changed file paths to lint (when empty, analyzer is a no-op)")
Analyzer.Flags.StringVar(&changedFilesCSV, "changed-files", "", "comma-separated list of changed file paths to lint (when empty, analyzer is a no-op; use \"all\" to audit every file)")
Analyzer.Flags.BoolVar(&fullRepo, "full-repo", false, "audit every analyzed file instead of only the changed ones")
}

func run(pass *analysis.Pass) (any, error) {
if fullRepo || isFullRepoSentinel(changedFilesCSV) {
pkgLog.Printf("analyzing package %s in full-repo mode", pass.Pkg.Path())
return runOnFiles(pass, nil)
}

changed := parseChangedFiles(changedFilesCSV)
if len(changed) == 0 {
pkgLog.Printf("no changed files provided for %s, skipping", pass.Pkg.Path())
return nil, nil
}
pkgLog.Printf("analyzing package %s (%d changed files)", pass.Pkg.Path(), len(changed))

return runOnFiles(pass, changed)
}

// runOnFiles analyzes the package. When changed is nil every file is checked
// (full-repo audit mode); otherwise only files present in changed are checked.
func runOnFiles(pass *analysis.Pass, changed map[string]struct{}) (any, error) {
noLintIndex, err := nolint.Index(pass)
if err != nil {
return nil, err
Expand Down Expand Up @@ -92,7 +111,18 @@ func parseChangedFiles(csv string) map[string]struct{} {
return changed
}

// isFullRepoSentinel reports whether the -changed-files value requests a
// full-repository audit.
func isFullRepoSentinel(csv string) bool {
return strings.EqualFold(strings.TrimSpace(csv), fullRepoSentinel)
}

// shouldCheckFile reports whether filename is in scope. A nil changed set means
// full-repo audit mode, where every file is in scope.
func shouldCheckFile(filename string, changed map[string]struct{}) bool {
if changed == nil {
return true
}
path := filepath.ToSlash(filename)
for changedPath := range changed {
if path == changedPath || strings.HasSuffix(path, "/"+changedPath) {
Expand Down
22 changes: 22 additions & 0 deletions pkg/linters/errormessage/errormessage_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,25 @@ func TestErrorMessage(t *testing.T) {

analysistest.Run(t, analysistest.TestData(), errormessage.Analyzer, "errormessage")
}

func TestErrorMessageFullRepoFlag(t *testing.T) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The test runs on the same testdata/errormessage fixture package as the baseline TestErrorMessage, so it does not verify that full-repo mode checks additional files beyond the -changed-files selection — it cannot distinguish scope expansion from the diff-gated path.

💡 Suggested fix

Create a separate fixture sub-package (e.g. testdata/errormessage/fullrepopkg/) that contains a violation and is not listed in any -changed-files flag. Run the full-repo tests against that sub-package and confirm the violation surfaces:

func TestErrorMessageFullRepoFlag(t *testing.T) {
    // fullrepopkg contains a violation not covered by the diff-gated path
    analysistest.Run(t, analysistest.TestData(), errormessage.Analyzer, "errormessage/fullrepopkg")
}

Without this, the two new tests only confirm the code compiles and branches correctly, not that the full-repo scope expansion actually works end-to-end.

@copilot please address this.

if err := errormessage.Analyzer.Flags.Set("full-repo", "true"); err != nil {
t.Fatalf("failed to set full-repo flag: %v", err)
}
t.Cleanup(func() {
_ = errormessage.Analyzer.Flags.Set("full-repo", "false")
})

analysistest.Run(t, analysistest.TestData(), errormessage.Analyzer, "errormessage")
}

func TestErrorMessageFullRepoSentinel(t *testing.T) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] In TestErrorMessageFullRepoSentinel, the cleanup only resets changed-files but leaves full-repo at its default. However, if TestErrorMessageFullRepoFlag runs first and its cleanup fails (panic before registration), full-repo could be left as true, making this test pass for the wrong reason. Register cleanup immediately after t.Fatalf calls — before any mutating operation — to guard against this.

💡 Recommended pattern
func TestErrorMessageFullRepoSentinel(t *testing.T) {
    t.Cleanup(func() { _ = errormessage.Analyzer.Flags.Set("changed-files", "") })
    if err := errormessage.Analyzer.Flags.Set("changed-files", "all"); err != nil {
        t.Fatalf("failed to set changed-files flag: %v", err)
    }
    analysistest.Run(t, analysistest.TestData(), errormessage.Analyzer, "errormessage")
}

Registering cleanup first ensures it runs even if Set succeeds but a later step panics.

@copilot please address this.

if err := errormessage.Analyzer.Flags.Set("changed-files", "all"); err != nil {
t.Fatalf("failed to set changed-files flag: %v", err)
}
t.Cleanup(func() {
_ = errormessage.Analyzer.Flags.Set("changed-files", "")
})

analysistest.Run(t, analysistest.TestData(), errormessage.Analyzer, "errormessage")
}