Skip to content

fix: preserve multi-stream projection order across batches - #707

Open
jlabedo wants to merge 3 commits into
ecotoneframework:mainfrom
jlabedo:codex/fix-multi-stream-projection-order
Open

jlabedo wants to merge 3 commits into
ecotoneframework:mainfrom
jlabedo:codex/fix-multi-stream-projection-order

Conversation

@jlabedo

@jlabedo jlabedo commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Why is this change proposed?

PDO projections that read multiple streams can process newer events before older events left in another stream’s next batch. EventStoreGlobalStreamSource::loadFromMultipleStreams limited each stream separately, then sorted only the events already loaded.

With a batch size of 2, the projection regression test received 1…6, 20, 21, 7…12 instead of 1…12, 20, 21.

Description of Changes

Summary

Merge the next event from each stream using bounded buffers:

- load one batch per stream and sort all loaded events
- advance each cursor to the last loaded event
+ compare stream heads and refill exhausted buffers before selecting the next event
+ return at most the requested number of events
+ advance each cursor only for returned events

Preserve event position order within each stream and use stream declaration order to resolve equal timestamps. Apply gap tracking and cleanup to the positions actually consumed.

Evidence

  • Before: all six new regression tests were observed failing before the fix, including test_projecting_multiple_streams_in_order_across_batches, which reproduces the ordering inversion between pages.
    After: the same tests pass, covering cursor resumption, buffer refills, equal timestamps, stream position order, and gaps loaded ahead of consumption.
  • Targeted multi-stream projection and gap tests: 15 tests, 43 assertions.
  • Full PdoEventSourcing suite on PostgreSQL: 269 tests, 747 assertions, no failures; one skipped test and one deprecation in laminas/laminas-code.
  • Full PdoEventSourcing suite on MySQL: 269 tests, 745 assertions, no failures; two skipped tests and the same dependency deprecation.
  • Package PHPStan, PHP CS Fixer for the changed files, license checks, and git diff --check: passed.

Merge Danger

Door: two-way. The change can be reverted without a migration; the position format remains compatible.

Blast Radius: projections. PDO projection pages now contain at most the requested number of events. Multi-stream projections may perform additional reads when one stream contains most of the oldest events, while keeping buffers bounded.

Pull Request Contribution Terms

  • I have read and agree to the contribution terms outlined in CONTRIBUTING.

@jlabedo jlabedo changed the title fix: préserver l’ordre des projections sur plusieurs streams fix: preserve multi-stream projection order across batches Oct 7, 2026
@jlabedo

jlabedo commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

The follow-up fix in 0962147e5 prevents a filled gap from being lost during end-of-page cleanup.

If maxGapOffset is reduced between reads, a resumed cursor can contain a filled gap outside the new offset window. When an earlier event from another stream occupies the current page, the gap event is prefetched but not delivered. Cleanup previously removed its position from the cursor, so subsequent pages never returned it. The fix preserves loaded, unconsumed gap positions until they are delivered.

The regression test was committed separately in 9662168ba: it failed because the second page was empty instead of containing the deferred event, then passed after the fix.

Validation:

  • PostgreSQL package suite: 270 tests, 750 assertions; one skipped test and one laminas-code deprecation.
  • MySQL targeted projection and gap tests: 28 tests, 69 assertions, passing.
  • Package PHPStan and PHP CS Fixer checks: passing.

@jlabedo
jlabedo requested a review from dgafka October 7, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant