Skip to content

stream: amortize writable buffer compaction - #65847

Open
gurgunday wants to merge 1 commit into
nodejs:mainfrom
gurgunday:perf/stream-amortize-writable-compaction
Open

stream: amortize writable buffer compaction#65847
gurgunday wants to merge 1 commit into
nodejs:mainfrom
gurgunday:perf/stream-amortize-writable-compaction

Conversation

@gurgunday

Copy link
Copy Markdown
Member

Context

Writable compacts its pending-write array after consuming more than 256 entries, even when most of the queue remains. Therefore draining a large backlog repeatedly shifts the remaining entries

Solution

Compact only once at least half the array has been consumed, keeping queue compaction amortized linear. Consumed entries are still cleared immediately

                                                      confidence   improvement   accuracy (*)    (**)   (***)
streams/writable-buffered-drain.js n=100 count=1024                  +11.54 %   ±20.95%  ±31.61%  ±50.44%
streams/writable-buffered-drain.js n=100 count=16384  ***            +13.38 %   ±1.93%  ±2.75%  ±4.00%
streams/writable-buffered-drain.js n=100 count=65536  ***           +107.78 %   ±5.53%  ±7.93%  ±11.62%

Signed-off-by: Gürgün Dayıoğlu <hey@gurgun.day>
Assisted-by: Codex
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/performance
  • @nodejs/streams

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. stream Issues and PRs related to Node.js streams. labels Sep 6, 2026
@gurgunday gurgunday added the performance Issues and PRs related to the performance of Node.js. label Sep 6, 2026

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@codecov

codecov Bot commented Sep 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 90.18%. Comparing base (b265340) to head (6c70f6e).
⚠️ Report is 26 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/streams/writable.js 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##             main   #65847    +/-   ##
========================================
  Coverage   90.18%   90.18%            
========================================
  Files         770      771     +1     
  Lines      264410   264622   +212     
  Branches    50230    50232     +2     
========================================
+ Hits       238449   238653   +204     
+ Misses      16966    16956    -10     
- Partials     8995     9013    +18     
Files with missing lines Coverage Δ
lib/internal/streams/writable.js 96.30% <0.00%> (ø)

... and 36 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gurgunday gurgunday added the request-ci Add this label to start a Jenkins CI on a PR. label Sep 6, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Sep 6, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. performance Issues and PRs related to the performance of Node.js. stream Issues and PRs related to Node.js streams.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants