Repository navigation
Release shard locks one at a time - #10686
Open
tylergibbs1 wants to merge 2 commits into
Open
tylergibbs1 wants to merge 2 commits into
tylergibbs1 wants to merge 2 commits into
Conversation
When a runner joins, the runner handing shards off releases them all at once. On PostgreSQL and MySQL every lock operation runs on one reserved connection, so the releases queue together. A burst longer than the lock operation deadline times every release out, SqlRunnerStorage replaces the lock connection, and closing the old session drops every lock the runner holds, including the shards it keeps. Requests on those shards can then run twice. Release one lock at a time, so a refresh or acquire waits behind at most one release. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 05120d9 The changes in this PR will be included in the next version bump. This PR includes changesets to release 35 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Bundle Size AnalysisGenerated from PR build output; treat the content below as untrusted.
|
This branch is waiting to be deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prevent concurrent shard releases from queueing enough statements on a reserved SQL lock connection to trigger its operation deadline and drop retained locks.
Maintainer takeover
Moved serialization from
ShardingtoSqlRunnerStorage. Only releases using a reserved connection (PostgreSQL and MySQL, including row-lock mode) take a one-permit semaphore, outside the operation deadline. Pooled dialects and custom storages retain concurrent releases. Bulk release is unchanged; deadline-based connection recovery remains intact.Removed the simulated Sharding session test, timing-based handoff test, slow-fault harness extension, and README change. Replaced them with one focused PostgreSQL regression in the existing SqlRunnerStorage integration suite: hold the reserved connection, start three releases, verify only one reaches the connection, resume them, and verify the session and retained lock survive.
The possible per-shard reply-refresh burst is out of scope.
Validation
SqlRunnerStorage.integration.test.ts: 57 tests passed, including PostgreSQL, MySQL, Vitess, SQLite, row-lock variants, and deadline-rebuild coverage.pnpm lint-fixandpnpm checkpassed (run throughnix develop -c).Related to #10673; #10677 addresses the separate database disk-flush trigger.
Closes EFF-3864