Skip to content

[v24.x backport] node:stream/iter and QUIC backports - #64675

Open
aduh95 wants to merge 118 commits into
nodejs:v24.x-stagingfrom
aduh95:stream-iter-and-quic-backports
Open

[v24.x backport] node:stream/iter and QUIC backports#64675
aduh95 wants to merge 118 commits into
nodejs:v24.x-stagingfrom
aduh95:stream-iter-and-quic-backports

Conversation

@aduh95

@aduh95 aduh95 commented Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Backport of:

@aduh95
aduh95 requested a review from a team as a code owner July 22, 2026 11:54
@aduh95 aduh95 added the semver-minor PRs that contain new features and should be released in the next minor version. label Jul 22, 2026
@nodejs-github-bot

nodejs-github-bot commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/actions
  • @nodejs/performance
  • @nodejs/releasers
  • @nodejs/security-wg
  • @nodejs/v8-update

@nodejs-github-bot nodejs-github-bot added the meta Issues and PRs related to the general management of the project. label Jul 22, 2026
@aduh95
aduh95 marked this pull request as draft July 22, 2026 11:55
@aduh95
aduh95 force-pushed the stream-iter-and-quic-backports branch from 935ead8 to 1a1c988 Compare July 22, 2026 11:57
@aduh95
aduh95 marked this pull request as ready for review July 22, 2026 11:57
@aduh95
aduh95 marked this pull request as draft July 22, 2026 11:58
@aduh95
aduh95 force-pushed the stream-iter-and-quic-backports branch from 1a1c988 to 975ed7b Compare July 22, 2026 11:58
@aduh95
aduh95 marked this pull request as ready for review July 22, 2026 11:59
@aduh95
aduh95 marked this pull request as draft July 22, 2026 11:59
@aduh95
aduh95 force-pushed the stream-iter-and-quic-backports branch from 975ed7b to 2b26bbe Compare July 22, 2026 12:00
@aduh95
aduh95 marked this pull request as ready for review July 22, 2026 12:00
@aduh95
aduh95 force-pushed the stream-iter-and-quic-backports branch 4 times, most recently from b28dcf7 to 3c553ff Compare July 22, 2026 13:32
@aduh95
aduh95 changed the base branch from v24.19.0-proposal to v24.x-staging July 22, 2026 13:33
@aduh95
aduh95 marked this pull request as draft July 22, 2026 13:33
@aduh95
aduh95 marked this pull request as ready for review July 22, 2026 13:33
@aduh95
aduh95 force-pushed the stream-iter-and-quic-backports branch from 3c553ff to 173b666 Compare July 22, 2026 13:37
@aduh95
aduh95 force-pushed the stream-iter-and-quic-backports branch 3 times, most recently from 7e7fa1a to 182db6c Compare July 22, 2026 15:53
@codecov

codecov Bot commented Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.23%. Comparing base (b881658) to head (65439dd).
⚠️ Report is 432 commits behind head on v24.x-staging.

Additional details and impacted files 
@@                Coverage Diff                @@
##           v24.x-staging   #64675      +/-   ##
=================================================
+ Coverage          89.92%   90.23%   +0.31%     
=================================================
  Files                686      710      +24     
  Lines             208389   226450   +18061     
  Branches           40077    42776    +2699     
=================================================
+ Hits              187387   204341   +16954     
- Misses             13238    14100     +862     
- Partials            7764     8009     +245

see 273 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@aduh95
aduh95 force-pushed the stream-iter-and-quic-backports branch 2 times, most recently from c1ef609 to 421ba7e Compare July 24, 2026 09:13
@nodejs-github-bot

nodejs-github-bot commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/75090/

aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@nodejs-github-bot @aduh95
deps: update ngtcp2 to 1.22.0
080c4d7 
PR-URL: #62595
Backport-PR-URL: #64675
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@jasnell @aduh95
quic: continue working on quic api bits
54c094d 
PR-URL: #60123
Backport-PR-URL: #64675
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@sravani1510 @aduh95
test: skip quic tests that IBM i does not support
2cc54e1 
PR-URL: #60160
Backport-PR-URL: #64675
Reviewed-By: Richard Lau <richard.lau@ibm.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@addaleax @aduh95
src: show original file name in FileHandle GC close errors
fbb3960 
Otherwise, it can be virtually impossible to debug where these types
of errors originate from.

PR-URL: #60593
Backport-PR-URL: #64675
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@jasnell @aduh95
stream: add stream/iter Implementation
28dc85d 
Experimental implementation of https://stream-iter.jasnell.me/

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-By: Claude/Opus 4.6
PR-URL: #62066
Backport-PR-URL: #64675
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@jasnell @aduh95
test: add tests for experimental stream/iter implementation
14c5473 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-By: Claude/Opus 4.6
PR-URL: #62066
Backport-PR-URL: #64675
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@jasnell @aduh95
benchmark: add benchmarks for experimental stream/iter
a290f51 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-By: Claude/Opus 4.6
PR-URL: #62066
Backport-PR-URL: #64675
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@jasnell @aduh95
stream: add stream/iter to classic stream adapters
baf98fa 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode/Opus 4.6
PR-URL: #62469
Backport-PR-URL: #64675
Reviewed-By: Robert Nagy <ronagy@icloud.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@watilde @aduh95
doc: fix doubled word typo in stream_iter.md
630b15e 
PR-URL: #62916
Backport-PR-URL: #64675
Reviewed-By: Richard Lau <richard.lau@ibm.com>
Reviewed-By: Deokjin Kim <deokjin81.kim@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@Renegade334 @aduh95
doc: minor structural stream/iter edits
859a02d 
Signed-off-by: Renegade334 <contact.9a5d6388@renegade334.me.uk>
PR-URL: #63089
Backport-PR-URL: #64675
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@Renegade334 @aduh95
stream: remove redundant method check from iter.pipeToSync
274b3a2 
Signed-off-by: Renegade334 <contact.9a5d6388@renegade334.me.uk>
PR-URL: #63099
Backport-PR-URL: #64675
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@Renegade334 @aduh95
stream: minor stream/iter implementation edits
b57b51e 
Signed-off-by: Renegade334 <contact.9a5d6388@renegade334.me.uk>
PR-URL: #63132
Backport-PR-URL: #64675
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: cache minimum cursor count in share
eb72fe9 
Track the number of consumers at the cached minimum cursor in share()
so the minimum is only recomputed when the last consumer at that cursor
advances or detaches.

This avoids scanning every consumer on each trim attempt when multiple
consumers advance through a shared buffer.

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63262
Backport-PR-URL: #64675
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: René <contact.9a5d6388@renegade334.me.uk>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: preserve toReadableSync batch after backpressure
9a45ff5 
Keep the current batch and index across _read() calls so chunks that
remain after push() returns false are emitted on later reads.

Fixes: #63275

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63276
Backport-PR-URL: #64675
Fixes: #63275
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: optimize single-slot push queue drain
c98cc67 
Avoid allocating a new result array when PushQueue drains a single
queued slot. Return that slot directly and keep the existing flattening
path for multiple queued slots.

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63274
Backport-PR-URL: #64675
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: validate fromWritable() options before cache
ae79f2b 
Validate options before returning a cached fromWritable() adapter so
invalid later options still throw.

Cache adapters by backpressure policy as well as Writable instance,
since the policy changes write behavior.

Fixes: #63277

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63278
Backport-PR-URL: #64675
Fixes: #63277
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: uncork fromWritable writev on chunk error
944470b 
Ensure fromWritable().writev() uncorks the wrapped Writable when
converting a later chunk throws. This prevents an internal cork from
leaking after ERR_INVALID_ARG_TYPE.

Fixes: #63294

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63295
Backport-PR-URL: #64675
Fixes: #63294
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: validate broadcast writer writev chunks
f63143f 
Validate BroadcastWriter writev() and writevSync() inputs before
converting chunks so non-array values throw ERR_INVALID_ARG_TYPE.

Fixes: #63299

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63300
Backport-PR-URL: #64675
Fixes: #63299
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: avoid retrying accepted pipeTo writes
ec2666e 
PushWriter in block backpressure mode can return false from writeSync()
and writevSync() after accepting data. Treat that false return as
backpressure and wait for drain instead of retrying the same chunks
asynchronously.

Fixes: #63296

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63297
Backport-PR-URL: #64675
Fixes: #63296
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: cache minimum cursor count in broadcast
1d7c86d 
Track how many broadcast consumers share the cached minimum cursor,
matching the share implementation. This lets buffer trimming avoid
calling getMinCursor() until the last minimum-cursor consumer
advances or detaches. Add coverage for fan-out trimming behavior.

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63322
Backport-PR-URL: #64675
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: limit iter from sync iterable batches
c50d8e4 
Bound sync iterable normalization in from() and fromSync() to
FROM_BATCH_SIZE. This avoids unbounded batches for from() sync
iterable fallbacks and lets fromSync() coalesce plain Uint8Array
values for writev paths.

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63324
Backport-PR-URL: #64675
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: fix merge handling for object-like sources
4c9f3ed 
merge() treated any final non-iterable object as an options object.
That dropped valid from() inputs such as ArrayBuffer, ArrayBufferView,
and streamable protocol objects.

Fixes: #63355

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63356
Backport-PR-URL: #64675
Fixes: #63355
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: René <contact.9a5d6388@renegade334.me.uk>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: add sync iterable fast path to pipeTo
02a51a7 
Avoid normalizing sync iterable sources through from() when pipeTo()
has no transforms or signal and the writer can accept sync writes.
This keeps writes incremental while preserving async fallback for
values that still need it.

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63318
Backport-PR-URL: #64675
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
benchmark: respect stream/iter broadcast backpressure
b022a14 
Only decrement the remaining byte count after a stream/iter broadcast
write is accepted. If writeSync() is blocked by strict backpressure,
fall back to the async write() path for the same chunk.

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63314
Backport-PR-URL: #64675
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
aduh95 added a commit that referenced this pull request Aug 6, 2026
@aduh95
doc: fix promise nomenclature in stream_iter.md
cc425bc 
Signed-off-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
PR-URL: #63406
Backport-PR-URL: #64675
Reviewed-By: René <contact.9a5d6388@renegade334.me.uk>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@ikeyan @aduh95
stream: align Readable.toWeb termination with eos
5658c63 
PR-URL: #62394
Backport-PR-URL: #64675
Reviewed-By: Mattias Buelens <mattias@buelens.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@Renegade334 @aduh95
stream: disallow writing string chunk with 'buffer' encoding
fce3df7 
Signed-off-by: Renegade334 <contact.9a5d6388@renegade334.me.uk>
PR-URL: #63062
Backport-PR-URL: #64675
Refs: #33075
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: propagate abort reason in share and broadcast
88a3392 
Pass signal.reason to the multi-consumer cancel paths so signal abort
is reported as AbortError instead of clean iterator completion.

Also make detached share consumers rethrow a stored source error when
they resume after cancellation, preserving the abort reason for pending
pulls.

Fixes: #63357

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63358
Backport-PR-URL: #64675
Fixes: #63357
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: avoid duplicate writes in toWritable
2e86855 
PushWriter can return false after accepting a chunk when block
backpressure is active. Teach the classic Writable adapter to treat
that case as accepted backpressure instead of retrying through the
async write path.

Fixes: #63359

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63360
Backport-PR-URL: #64675
Fixes: #63359
Reviewed-By: James M Snell <jasnell@gmail.com>
aduh95 pushed a commit that referenced this pull request Aug 6, 2026
@trivikr @aduh95
stream: flush each fused stateless transform
217b495 
Ensure consecutive stateless stream/iter transforms each receive a
final null flush after upstream flush output has been processed.

Fixes: #63467

Signed-off-by: Kamat, Trivikram <16024985+trivikr@users.noreply.github.com>
Assisted-by: openai:gpt-5.5
PR-URL: #63468
Backport-PR-URL: #64675
Fixes: #63467
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jasnell jasnell Awaiting requested review from jasnell

Assignees

No one assigned

Labels

meta Issues and PRs related to the general management of the project. semver-minor PRs that contain new features and should be released in the next minor version.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.