fix(logstash source): preserve writer windows when generating ACKs#25531
Open
bruceg wants to merge 2 commits into
Open
fix(logstash source): preserve writer windows when generating ACKs#25531bruceg wants to merge 2 commits into
bruceg wants to merge 2 commits into
Conversation
bruceg
added
type: bug
github-actions
Bot
added
the
domain: sources
Filebeat expects ACKs to remain within the current Lumberjack writer window. The logstash source decoder was ignoring WindowSize frames and the receiver could later batch frames from multiple windows together before building an ACK. When that happened, Vector could ACK the highest sequence in the merged batch instead of the last sequence in the current window. That behavior could surface as "invalid sequence number received" on the sender side and could contribute to reconnects and duplicate retransmits under load. Fix this by preserving writer window boundaries during decode, tracking which decoded frames close a window, and emitting one ACK frame per completed window when a batched read contains multiple windows. This keeps the existing batching behavior in the generic TCP path while making the logstash receiver respect the expected ACK semantics. Also add unit tests that demonstrate the bug with both sequence resets and monotonic sequences across adjacent windows.
bruceg
force-pushed
the
bruceg/fix-logstash-pipelining
branch
from
7ee8e13 to
b68bf9f
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7ee8e13e03
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Member
Author
|
@codex review |
|
Codex Review: Didn't find any major issues. 👍 ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Filebeat expects ACKs to remain within the current Lumberjack writer window. The logstash source decoder was ignoring WindowSize frames and the receiver could later batch frames from multiple windows together before building an ACK. When that happened, Vector could ACK the highest sequence in the merged batch instead of the last sequence in the current window.
That behavior has resulted "invalid sequence number received" errors on the sender side and results in reconnects and duplicate retransmits under load.
Fix this by preserving writer window boundaries during decode, tracking which decoded frames close a window, and emitting one ACK frame per completed window when a batched read contains multiple windows. This keeps the existing batching behavior in the generic TCP path while making the logstash receiver respect the expected ACK semantics.
Also add unit tests that demonstrate the bug with both sequence resets and monotonic sequences across adjacent windows.
Vector configuration
N/A
How did you test this PR?
Unit tests included. I tried to test using Filebeat itself, but triggering this is highly timing dependent.
Change Type
Is this a breaking change?
Does this PR include user facing changes?
no-changeloglabel to this PR.References
Notes
@vectordotdev/vectorto reach out to us regarding this PR.pre-pushhook, please see this template.make fmtmake check-clippy(if there are failures it's possible some of them can be fixed withmake clippy-fix)make testgit merge origin masterandgit push.Cargo.lock), pleaserun
make build-licensesto regenerate the license inventory and commit the changes (if any). More details on the dd-rust-license-tool.