Skip to content

feat(error-handler): add ErrorHandlerInterceptor plugin - #262

Open
rossaddison wants to merge 6 commits into
php-testo:1.xfrom
rossaddison:feat/73-error-handler-interceptor
Open

feat(error-handler): add ErrorHandlerInterceptor plugin#262
rossaddison wants to merge 6 commits into
php-testo:1.xfrom
rossaddison:feat/73-error-handler-interceptor

Conversation

@rossaddison

Copy link
Copy Markdown
Contributor

Closes #73

Summary

  • New plugin/error-handler/ package (testo/error-handler)
  • ErrorHandlerPlugin — the PluginConfigurator users add to their ApplicationConfig
  • ErrorHandlerInterceptor — wraps each test in set_error_handler() / restore_error_handler() at ORDER_CLOSE_TO_TEST priority
  • CapturedError — value object for a single PHP error (severity, message, file, line)
  • CapturedErrors — collection stored as a TestResult attribute under CapturedErrors::class

Behaviour

Mode What happens
failOnError: false (default) Errors are captured; test status is unchanged; renderers can inspect the CapturedErrors attribute
failOnError: true A captured error upgrades a passing test to Status::Failed, wrapping the first error in an ErrorException; already-failed/errored tests are not overridden

In both modes:

  • restore_error_handler() is called in a finally block, so the previous handler is always restored even when the test throws
  • All errors raised during the test are accumulated (not just the first)

Tests (10, all passing)

Test Covers
noErrorsPassesResultThrough No errors → result returned unmodified, no attribute attached
capturedErrorIsStoredAsAttribute Single error stored with correct message and severity
multipleErrorsAreAllCaptured All triggered errors appear in the list in order
collectModePreservesPassingStatus Default mode does not change Status::Passed
failModeUpgradesPassingTestToFailed failOnError: true sets Status::Failed + ErrorException failure
failModeUsesFirstErrorAsFailure First error is the one wrapped, not the last
failModeDoesNotOverrideAlreadyFailedTest Pre-existing Status::Failed + original throwable are preserved
failModeDoesNotOverrideErrorStatus Pre-existing Status::Error is preserved
handlerIsRestoredAfterTestCompletes Outer handler is active again after a normal test run
handlerIsRestoredEvenWhenTestThrows Outer handler is active again even when the test pipeline throws

Monorepo wiring

  • composer.jsonrequire, autoload-dev, path-repository version entry
  • testo.php — src exclusion + suites.php include
  • .github/workflows/split-publish.ymlerror-handler-[0-9]* tag for subtree split

Note on set_error_handler callback signatures

The two outer-handler closures in the test use zero parameters (static function () use (&$count)) rather than (int $errno, string $errstr). PHP silently discards extra arguments when a callable declares fewer params than the caller passes — this is standard PHP behaviour. The zero-param form avoids SonarQube S1172 (unused parameter) without needing @SuppressWarnings or the $_-prefix convention.

@rossaddison
rossaddison requested a review from a team as a code owner July 6, 2026 13:48
@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@rossaddison

Copy link
Copy Markdown
Contributor Author

Implementation summary — testo/error-handler plugin

What this adds

A new plugin/error-handler/ package that wraps every test in set_error_handler() / restore_error_handler(), accumulating any PHP errors triggered during the test into a CapturedErrors attribute on the returned TestResult.

Files

File Purpose
src/CapturedError.php Value object — severity, message, file, line for one PHP error
src/Internal/CapturedErrors.php Collection stored as a TestResult attribute under CapturedErrors::class
src/Internal/ErrorHandlerInterceptor.php TestRunInterceptor at ORDER_CLOSE_TO_TEST priority — installs/restores handler, attaches attribute, optionally fails the test
src/ErrorHandlerPlugin.php PluginConfigurator users add to their ApplicationConfig
tests/Unit/ErrorHandlerInterceptorTest.php 10 unit tests (all passing)

Behaviour

Mode Result
new ErrorHandlerPlugin() (default) Errors collected; test status unchanged; renderers may inspect CapturedErrors attribute
new ErrorHandlerPlugin(failOnError: true) First captured error upgrades a passing test to Status::Failed with an ErrorException; pre-existing Failed/Error results are not overridden

restore_error_handler() is always called in a finally block — the previous handler is restored even when the test pipeline throws.

Test coverage

Test What it verifies
noErrorsPassesResultThrough No errors → result unmodified, no attribute
capturedErrorIsStoredAsAttribute Single error → correct message and severity
multipleErrorsAreAllCaptured All triggered errors appear in order
collectModePreservesPassingStatus Default mode keeps Status::Passed
failModeUpgradesPassingTestToFailed failOnError: trueStatus::Failed + ErrorException
failModeUsesFirstErrorAsFailure First error wins as the failure, not the last
failModeDoesNotOverrideAlreadyFailedTest Pre-existing Status::Failed + original throwable preserved
failModeDoesNotOverrideErrorStatus Pre-existing Status::Error preserved
handlerIsRestoredAfterTestCompletes Outer handler is active again after a normal run
handlerIsRestoredEvenWhenTestThrows Outer handler is active again even when the pipeline throws

Note on zero-param closures in tests

The outer-handler closures in the restoration tests use static function () use (&$count) with no declared parameters. PHP silently discards extra arguments when a callable declares fewer params than the caller passes — standard PHP behaviour — and the zero-param form avoids SonarQube S1172 (unused parameter) without needing suppress annotations or $_-prefix workarounds.

Monorepo wiring

  • composer.jsonrequire, autoload-dev, path-repository version entry
  • testo.php — src exclusion + suites.php include
  • .github/workflows/split-publish.ymlerror-handler-[0-9]* tag for subtree split on release

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new testo/error-handler plugin package that intercepts PHP errors during test execution, records them on TestResult, and optionally turns captured errors into test failures—integrating it into the monorepo (Composer wiring, suite registration, and split-publish tagging).

Changes:

  • Introduces ErrorHandlerInterceptor + value objects (CapturedError, CapturedErrors) to capture and persist PHP errors raised during a test run.
  • Adds an ErrorHandlerPlugin configurator to register the interceptor (with a failOnError option).
  • Wires the new plugin into the monorepo: root Composer requirement + path version mapping, Testo suites inclusion, Infection config tweak, and split-publish tag pattern.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/Application/Stub/EmptyRun/.placeholder.php Adds an empty fixture file to ensure the stub directory exists in mirrored test layouts.
testo.php Registers the new plugin test suite location and includes its suites.php.
plugin/error-handler/tests/Unit/ErrorHandlerInterceptorTest.php Adds unit tests for capturing/accumulating errors and handler restoration behavior.
plugin/error-handler/tests/suites.php Declares the ErrorHandler unit test suite for Testo’s suite discovery.
plugin/error-handler/src/Internal/ErrorHandlerInterceptor.php Implements the interceptor that installs/restores an error handler and records errors on the result.
plugin/error-handler/src/Internal/CapturedErrors.php Adds the collection type used to store captured errors on TestResult.
plugin/error-handler/src/ErrorHandlerPlugin.php Adds the public plugin configurator that registers the interceptor.
plugin/error-handler/src/CapturedError.php Adds the value object representing a single captured PHP error.
plugin/error-handler/composer.json Introduces the standalone package metadata and autoloading for the new plugin.
infection.json Updates Infection mutator configuration to ignore inline-test source patterns.
composer.json Adds the new plugin to root dependencies and monorepo package version mapping/autoload-dev.
.github/workflows/split-publish.yml Extends subtree split publishing to recognize error-handler-* tags.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread plugin/error-handler/src/Internal/ErrorHandlerInterceptor.php Outdated
Comment on lines +12 to +20
/**
* Plugin that captures PHP errors raised during test execution.
*
* By default errors are collected and stored as a {@see Internal\CapturedErrors} attribute
* on the {@see \Testo\Core\Context\TestResult}, but the test still passes. Pass
* {@see $failOnError}: true to make any captured error fail the test instead.
*
* @api
*/

@roxblnfk roxblnfk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the time you're putting in.
I see you're leaning on AI heavily. So am I — but opening raw AI output as a PR without a human review pass is dismissive of the reviewer's time.

This PR has the same defect as #254: process-global state that isn't fiber-safe. The error-handler stack is process-global and tests can run inside fibers — when a test suspends, its handler stays on the stack while another runs, so errors leak into the wrong test's CapturedErrors, and interleaved resume makes restore_error_handler() pop the wrong frame. The fix is the scope-swap already in MockeryInterceptor::run() / MessengerHub::scope(): inside a fiber, restore the previous handler on suspend and re-install the test's on resume.

A second, open question: a test may change the error handler itself during the run. The interceptor should notice that and react — but the right behavior isn't obvious, so it's worth researching and discussing before implementing.

If you want to contribute to the project, I ask you to:

  • carry the fixes from earlier reviews of your PRs into future ones;
  • understand the issue first, and ask questions before implementing if you have any;
  • validate the AI agent's output yourself. That part is the contributor's, not the reviewer's.

@roxblnfk
roxblnfk marked this pull request as draft July 24, 2026 10:38
rossaddison and others added 6 commits August 13, 2026 17:32
…hp-testo#159)

When Infection mutates values inside a #[TestInline(arguments: [...],
result: ...)] attribute it is mutating test data, not production logic.
Every such mutation is "killed" (Assert::same catches the wrong expected
value), but the kills are semantically meaningless — they verify that the
assertion works, not that the source code handles a mutation correctly.
This inflates the mutation score with noise and wastes CI runner time.

Add `global-ignoreSourceCodeByRegex` to infection.json so Infection skips
any mutation whose source line contains `#[TestInline`. This covers all
single-line attribute declarations in both the Self-test fixtures under
plugin/inline/tests/Self/ and any production code that uses the attribute.
Method bodies on adjacent lines are unaffected and continue to be mutated.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…is mirrored

tests/Application/Stub/EmptyRun/ is intentionally empty — it is the test
fixture for EmptyRunTest, which asserts that a Testo run over an empty
directory yields Status::Risky with zero tests collected.

Git does not track empty directories, and bin/build-phpunit.php only copies
*.php files when populating the tests/PhpUnit/ mirror, so the mirror never
contained tests/PhpUnit/Application/Stub/EmptyRun/. The mirrored EmptyRunTest
resolved __DIR__ . '/../../Stub/EmptyRun' to that missing path and threw
InvalidArgumentException: File or directory not found — aborting Infection's
initial PHPUnit test run on every CI push to 1.x.

Add .placeholder.php (no namespace, no classes, no tests) to the source
directory. The build script copies it verbatim into the mirror, which creates
the required directory. Testo's FinderConfig still discovers zero tests there,
so Status::Risky is reported and the assertion holds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…urceCodeByRegex

Specifying "mutators": {} without "@default" treats the block as an
allowlist, so all default mutators were silently disabled — producing
0 mutations and an MSI failure.  Adding "@default": true restores the
full default mutator set while the global regex filter still skips
lines containing #[TestInline.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements the error handler interceptor described in issue php-testo#73.

The plugin wraps each test in set_error_handler() / restore_error_handler()
and accumulates any PHP errors triggered during the test into a CapturedErrors
attribute on the returned TestResult.

Behaviour:
- Default (failOnError: false): errors are collected and stored as a
  CapturedErrors attribute; the test result status is unchanged.
- failOnError: true: a captured error upgrades a passing test to
  Status::Failed and wraps the first error in an ErrorException as the
  failure, preserving any pre-existing failure from the next() chain.

Includes 10 unit tests covering collect mode, fail mode, multiple errors,
first-error-wins semantics, and handler restoration (both normal and throw
paths). All tests use zero-param closures for set_error_handler callbacks to
avoid SonarQube S1172 (unused parameter) — PHP silently discards extra
arguments when a callable declares fewer params than the caller passes.

Also wires the plugin into the monorepo: composer.json (require + autoload-dev
+ path-repository version), testo.php (src exclusion + suites), and
split-publish.yml (error-handler-[0-9]* tag).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…CapturedErrors to public

Addresses roxblnfk's review on php-testo#262:

- set_error_handler()/restore_error_handler() operate on one process-global
  stack. The old code installed its handler once before $next() and
  restored once after — but $next() can suspend a fiber mid-test while a
  sibling test interleaves, so the handler stayed installed (and, on an
  interleaved resume, restore_error_handler() could pop a sibling's frame
  instead of its own). Same defect as php-testo#254.

  Fixed by wrapping $next() in its own fiber and swapping the handler on
  every suspend/resume — restore (native stack pop) on suspend, reinstall
  on resume — mirroring the already-reviewed pattern in
  MockeryInterceptor::run() and MessengerHub::scope().

  Regression test added (restoresTheOuterHandlerWhileSuspendedAndReinstallsItsOwnOnResume):
  confirmed it fails against the old code (an error fired while suspended
  was wrongly captured by this test's own handler instead of reaching the
  outer one) and passes against the fix.

- Promoted Internal\CapturedErrors to a public Testo\ErrorHandler\CapturedErrors
  class (Copilot review comment): the plugin's own docs already tell
  consumers to read this attribute off TestResult, so it was never really
  internal — it just wasn't marked as such. Fixes the @api-marked
  ErrorHandlerPlugin's docblock referencing an internal type.

- The other Copilot comment (restrict the failOnError status upgrade to
  Status::Passed) was already fixed in a prior commit on this branch — no
  change needed.

Also rebased onto current 1.x (26 commits behind), resolving one real
conflict in composer.json (version bumps landed upstream since this PR
opened) and the same CaseDefinition/CaseInfo required-argument fix already
applied on php-testo#264.

The second question from review — what should happen if a test changes
the error handler itself mid-run — is intentionally left open; per
roxblnfk's own comment it needs research/discussion before implementing,
not a quick fix.

Verified:
- composer rector:ci: clean, 0 files
- Full Testo suite: 1682 passed, 6 failed/7 error (same pre-existing
  Bench/Self baseline as the current rector/* PR series, unrelated)
- ErrorHandlerInterceptorTest: 11/11 passed, including the new
  fiber-safety regression test (confirmed it fails against the old code)
- Psalm: this repo's Psalm CI only covers core/ (confirmed via psalm.xml
  and psalm.yml's trigger paths) — plugin/error-handler/ was never in
  scope, unchanged by this fix

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@rossaddison
rossaddison force-pushed the feat/73-error-handler-interceptor branch from 78c4846 to e8a7008 Compare August 13, 2026 16:49
@rossaddison

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review — addressed both points, and validated the AI-generated fix myself rather than just pasting it.

Fiber-safety fix

ErrorHandlerInterceptor::runTest() now wraps $next() in its own fiber and swaps the error handler on every suspend/resume — restoring whichever handler was active before this test installed its own (the native set_error_handler() stack does that for free) on suspend, and reinstalling this test's handler on resume. This mirrors the pattern already reviewed and merged in MockeryInterceptor::run() and MessengerHub::scope().

Added a regression test, restoresTheOuterHandlerWhileSuspendedAndReinstallsItsOwnOnResume. I checked it's actually discriminating, not just green by construction: reverted to the old code locally, confirmed the new test fails against it (an error fired while the test is suspended gets wrongly swallowed by the still-installed handler instead of reaching the outer one), then restored the fix and confirmed it passes.

The second question you raised — what should happen if a test changes the error handler itself mid-run — I've left alone. You flagged it as needing research/discussion before implementing, so I'm not guessing at behavior here; happy to pick it up separately once there's a direction.

Copilot's comments

  • The internal-type-leak on ErrorHandlerPlugin's docblock: promoted Internal\CapturedErrors to a public Testo\ErrorHandler\CapturedErrors, matching its sibling CapturedError. The plugin's own docs already told consumers to read this attribute off TestResult, so it was never really internal — just not marked as such.
  • The failOnError status-upgrade restriction to Status::Passed was already fixed in an earlier commit on this branch.

Also

Rebased onto current 1.x (this had drifted 26 commits behind) — one real conflict in composer.json (version bumps landed upstream since this PR opened), resolved by taking the newer versions and re-adding the testo/error-handler require line.

Verification

  • composer rector:ci — clean, 0 files
  • Full Testo suite — 1682 passed, 6 failed/7 error (same pre-existing Bench/Self baseline as the other PRs in the current series, unrelated)
  • ErrorHandlerInterceptorTest — 11/11 passed, including the new fiber-safety test
  • Psalm — confirmed this repo's Psalm CI only covers core/ (via psalm.xml and psalm.yml's trigger paths); plugin/error-handler/ was never in scope, before or after this change

@rossaddison
rossaddison requested a review from roxblnfk August 13, 2026 16:54
@rossaddison
rossaddison marked this pull request as ready for review August 13, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error handler interceptor

3 participants