Skip to content

Throttle repeated packet-parse error logs in FullPacketParser - #179

Open
u9g wants to merge 1 commit into
ProtoDef-io:masterfrom
u9g:ratelimit-partial-read
Open

u9g wants to merge 1 commit into
ProtoDef-io:masterfrom
u9g:ratelimit-partial-read

Conversation

@u9g

@u9g u9g commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Problem

When a stream is malformed or version-mismatched, the same packet can fail to parse on every frame the server sends. FullPacketParser._transform logged the full stack on each PartialReadError, so one persistent fault produced thousands of identical stacks a minute and drowned every other log line. A real session was seen emitting ~15,000 identical Read error for undefined : undefined stacks — tens of megabytes — over ~45 minutes.

Change

A small per-instance throttle: each distinct error (keyed on the top line of its stack) is logged in full on its first occurrence, then again only at the 1st, 2nd, 4th, 8th, … occurrence with a repeat count. A persistent fault costs O(log n) lines instead of n, while the first full stack — the part you actually diagnose from — is always kept, and distinct faults are still each reported. The same throttle is applied to the chunk-size-mismatch log two lines up, which is the same failure family.

noErrorLogging still suppresses everything, unchanged.

Testing

  • 1,000 identical partial-read errors → 10 log lines (n = 1, 2, 4, … 512); two interleaved signatures × 4 each → 6 lines (each hits 1, 2, 4).
  • npm test — all 501 existing tests pass; parser semantics unchanged.

A malformed or mismatched stream can make the same packet fail to parse on
every frame it sends. FullPacketParser logged the full stack each time, so
one persistent fault produced thousands of identical stacks a minute and
buried every other log line -- a single session was seen writing ~15k
identical PartialReadError stacks, tens of megabytes.

Each distinct error (keyed on the top of its stack) is now logged in full
on its first occurrence, then only at the 1st, 2nd, 4th, 8th, ... with a
repeat count, so a persistent fault costs O(log n) lines instead of n. The
same throttle covers the chunk-size-mismatch log two lines up. Counts are
per parser instance; noErrorLogging still silences everything.

@rom1504 rom1504 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Astra agent review — AI-generated, not manually written by the maintainer.

The Astra agent reviewed this change at the maintainer's request. I inspected the current parser diff and ran the current FullPacketParser with a real ProtoDef schema and two distinct malformed buffers. The reproduction shows that only the first malformed packet retains its diagnostic details. I did not run the full suite.

Comment thread src/serializer.js
packet = this.parsePacketBuffer(chunk)
if (packet.metadata.size !== chunk.length && !this.noErrorLogging) {
console.log('Chunk size is ' + chunk.length + ' but only ' + packet.metadata.size + ' was read ; partial packet : ' +
logThrottled(this.errorCounts, 'chunk size mismatch',

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Astra agent review — AI-generated, not manually written by the maintainer.

The constant key combines every size mismatch in this parser, so a different malformed packet never gets its first full diagnostic. I reproduced this with a real one-byte container schema and buffers 01 63, 01 63, 02 58, 02 58: only the first packet's parsed value/buffer is logged; the second fault is silent at occurrence 3 and becomes just chunk size mismatch (repeated 4 times) at occurrence 4. Since these mismatches are only logged and the packets are still pushed downstream, the second fault's details are lost altogether. Could the grouping distinguish different packet/schema failures and preserve a first diagnostic for each, with bounded bookkeeping and an interleaved-malformed-packet test? The partial-read key has the same risk when different generated read paths share the first stack line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants