Conversation
A malformed or mismatched stream can make the same packet fail to parse on every frame it sends. FullPacketParser logged the full stack each time, so one persistent fault produced thousands of identical stacks a minute and buried every other log line -- a single session was seen writing ~15k identical PartialReadError stacks, tens of megabytes. Each distinct error (keyed on the top of its stack) is now logged in full on its first occurrence, then only at the 1st, 2nd, 4th, 8th, ... with a repeat count, so a persistent fault costs O(log n) lines instead of n. The same throttle covers the chunk-size-mismatch log two lines up. Counts are per parser instance; noErrorLogging still silences everything.
rom1504
left a comment
There was a problem hiding this comment.
Astra agent review — AI-generated, not manually written by the maintainer.
The Astra agent reviewed this change at the maintainer's request. I inspected the current parser diff and ran the current FullPacketParser with a real ProtoDef schema and two distinct malformed buffers. The reproduction shows that only the first malformed packet retains its diagnostic details. I did not run the full suite.
| packet = this.parsePacketBuffer(chunk) | ||
| if (packet.metadata.size !== chunk.length && !this.noErrorLogging) { | ||
| console.log('Chunk size is ' + chunk.length + ' but only ' + packet.metadata.size + ' was read ; partial packet : ' + | ||
| logThrottled(this.errorCounts, 'chunk size mismatch', |
There was a problem hiding this comment.
Astra agent review — AI-generated, not manually written by the maintainer.
The constant key combines every size mismatch in this parser, so a different malformed packet never gets its first full diagnostic. I reproduced this with a real one-byte container schema and buffers 01 63, 01 63, 02 58, 02 58: only the first packet's parsed value/buffer is logged; the second fault is silent at occurrence 3 and becomes just chunk size mismatch (repeated 4 times) at occurrence 4. Since these mismatches are only logged and the packets are still pushed downstream, the second fault's details are lost altogether. Could the grouping distinguish different packet/schema failures and preserve a first diagnostic for each, with bounded bookkeeping and an interleaved-malformed-packet test? The partial-read key has the same risk when different generated read paths share the first stack line.
Problem
When a stream is malformed or version-mismatched, the same packet can fail to parse on every frame the server sends.
FullPacketParser._transformlogged the full stack on eachPartialReadError, so one persistent fault produced thousands of identical stacks a minute and drowned every other log line. A real session was seen emitting ~15,000 identicalRead error for undefined : undefinedstacks — tens of megabytes — over ~45 minutes.Change
A small per-instance throttle: each distinct error (keyed on the top line of its stack) is logged in full on its first occurrence, then again only at the 1st, 2nd, 4th, 8th, … occurrence with a repeat count. A persistent fault costs O(log n) lines instead of n, while the first full stack — the part you actually diagnose from — is always kept, and distinct faults are still each reported. The same throttle is applied to the chunk-size-mismatch log two lines up, which is the same failure family.
noErrorLoggingstill suppresses everything, unchanged.Testing
npm test— all 501 existing tests pass; parser semantics unchanged.