worker: add worker.getHeapStatistics() by mcollina · Pull Request #57888 · nodejs/node

mcollina · 2025-04-15T10:28:31Z

Adds worker.getHeapStatistics() so that the heap usage of the worker could be observer from the parent thread.

mcollina · 2025-04-15T10:29:32Z

In node_v8.cc, the same is exposed via

Lines 57 to 71 in 2204587

    
           #define HEAP_STATISTICS_PROPERTIES(V)                                          \ 
        
             V(0, total_heap_size, kTotalHeapSizeIndex)                                   \ 
        
             V(1, total_heap_size_executable, kTotalHeapSizeExecutableIndex)              \ 
        
             V(2, total_physical_size, kTotalPhysicalSizeIndex)                           \ 
        
             V(3, total_available_size, kTotalAvailableSize)                              \ 
        
             V(4, used_heap_size, kUsedHeapSizeIndex)                                     \ 
        
             V(5, heap_size_limit, kHeapSizeLimitIndex)                                   \ 
        
             V(6, malloced_memory, kMallocedMemoryIndex)                                  \ 
        
             V(7, peak_malloced_memory, kPeakMallocedMemoryIndex)                         \ 
        
             V(8, does_zap_garbage, kDoesZapGarbageIndex)                                 \ 
        
             V(9, number_of_native_contexts, kNumberOfNativeContextsIndex)                \ 
        
             V(10, number_of_detached_contexts, kNumberOfDetachedContextsIndex)           \ 
        
             V(11, total_global_handles_size, kTotalGlobalHandlesSizeIndex)               \ 
        
             V(12, used_global_handles_size, kUsedGlobalHandlesSizeIndex)                 \ 
        
             V(13, external_memory, kExternalMemoryIndex)

. Do you want me to use that approach here?

Adds worker.getHeapStatistics() so that the heap usage of the worker could be observer from the parent thread. Signed-off-by: Matteo Collina <[email protected]>

Signed-off-by: Matteo Collina <[email protected]>

anonrig · 2025-04-15T11:00:28Z

src/node_worker.cc

+
+  v8::HeapStatistics heap_stats;
+  w->isolate_->GetHeapStatistics(&heap_stats);
+


If you use the object constructor that takes an array of keys and array of values, it will be faster

Have you got an example I can copy from?

Src/node_v8.cc line 350

Signed-off-by: Matteo Collina <[email protected]>

Qard

Only thing that might be of concern would be that placing this on the worker object loses the isolation as v8-specific that the one in the v8 module has. An alternative implementation could be something like v8.getWorkerHeapStatistics(worker) to keep it in that v8-specific segment of the API, if that's something we care about. I would not block on this, but I figured it worth bringing up.

Other than that, agreed that the array form object construction would be better.

codecov · 2025-04-15T11:58:41Z

Codecov Report

Attention: Patch coverage is 90.21739% with 9 lines in your changes missing coverage. Please review.

Project coverage is 90.23%. Comparing base (2204587) to head (df57fa3).
Report is 15 commits behind head on main.

Files with missing lines	Patch %	Lines
src/node_worker.cc	88.88%	5 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #57888      +/-   ##
==========================================
- Coverage   90.24%   90.23%   -0.01%     
==========================================
  Files         630      630              
  Lines      185688   185783      +95     
  Branches    36403    36414      +11     
==========================================
+ Hits       167569   167647      +78     
- Misses      10997    11009      +12     
- Partials     7122     7127       +5

Files with missing lines	Coverage Δ
lib/internal/worker.js	`99.82% <100.00%> (+<0.01%)`	⬆️
src/node_worker.h	`90.90% <ø> (ø)`
src/node_worker.cc	`84.45% <88.88%> (+0.72%)`	⬆️

... and 27 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

src/node_worker.cc

legendecas · 2025-04-15T12:29:03Z

Only thing that might be of concern would be that placing this on the worker object loses the isolation as v8-specific that the one in the v8 module has.

There is a precedent of Worker.getHeapSnapshot, coresponding to v8.getHeapSnapshot.

mcollina · 2025-04-15T13:23:16Z

Only thing that might be of concern would be that placing this on the worker object loses the isolation as v8-specific that the one in the v8 module has.

There is also the event loop utilization monitoring that can be done from the outside, too.

src/node_worker.cc

Signed-off-by: Matteo Collina <[email protected]>

mcollina · 2025-04-15T17:42:29Z

@joyeecheung @legendecas PTAL.

Signed-off-by: Matteo Collina <[email protected]>

nodejs-github-bot · 2025-04-16T15:34:43Z

CI: https://ci.nodejs.org/job/node-test-pull-request/66313/

nodejs-github-bot · 2025-04-17T05:42:45Z

CI: https://ci.nodejs.org/job/node-test-pull-request/66326/

nodejs-github-bot · 2025-04-17T10:31:00Z

Landed in 33d8e03

Adds worker.getHeapStatistics() so that the heap usage of the worker could be observer from the parent thread. Signed-off-by: Matteo Collina <[email protected]> PR-URL: #57888 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Chengzhong Wu <[email protected]> Reviewed-By: Darshan Sen <[email protected]> Reviewed-By: Stephen Belanger <[email protected]>

Notable changes: deps: * update timezone to 2025b (Node.js GitHub Bot) #57857 doc: * add dario-piotrowicz to collaborators (Dario Piotrowicz) #58102 * (SEMVER-MINOR) graduate multiple experimental apis (James M Snell) #57765 esm: * (SEMVER-MINOR) graduate import.meta properties (James M Snell) #58011 * (SEMVER-MINOR) support top-level Wasm without package type (Guy Bedford) #57610 sqlite: * (SEMVER-MINOR) add StatementSync.prototype.columns() (Colin Ihrig) #57490 src: * (SEMVER-MINOR) set default config as `node.config.json` (Marco Ippolito) #57171 * (SEMVER-MINOR) create `THROW_ERR_OPTIONS_BEFORE_BOOTSTRAPPING` (Marco Ippolito) #57016 * (SEMVER-MINOR) add config file support (Marco Ippolito) #57016 * (SEMVER-MINOR) add ExecutionAsyncId getter for any Context (Attila Szegedi) #57820 stream: * (SEMVER-MINOR) preserve AsyncLocalStorage context in finished() (Gürgün Dayıoğlu) #57865 util: * (SEMVER-MINOR) add `types.isFloat16Array()` (Livia Medeiros) #57879 worker: * (SEMVER-MINOR) add worker.getHeapStatistics() (Matteo Collina) #57888 PR-URL: #58388

Adds worker.getHeapStatistics() so that the heap usage of the worker could be observer from the parent thread. Signed-off-by: Matteo Collina <[email protected]> PR-URL: #57888 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Chengzhong Wu <[email protected]> Reviewed-By: Darshan Sen <[email protected]> Reviewed-By: Stephen Belanger <[email protected]>

Notable changes: deps: * update timezone to 2025b (Node.js GitHub Bot) #57857 doc: * add dario-piotrowicz to collaborators (Dario Piotrowicz) #58102 * (SEMVER-MINOR) graduate multiple experimental apis (James M Snell) #57765 esm: * (SEMVER-MINOR) graduate import.meta properties (James M Snell) #58011 * (SEMVER-MINOR) support top-level Wasm without package type (Guy Bedford) #57610 sqlite: * (SEMVER-MINOR) add StatementSync.prototype.columns() (Colin Ihrig) #57490 src: * (SEMVER-MINOR) set default config as `node.config.json` (Marco Ippolito) #57171 * (SEMVER-MINOR) create `THROW_ERR_OPTIONS_BEFORE_BOOTSTRAPPING` (Marco Ippolito) #57016 * (SEMVER-MINOR) add config file support (Marco Ippolito) #57016 * (SEMVER-MINOR) add ExecutionAsyncId getter for any Context (Attila Szegedi) #57820 stream: * (SEMVER-MINOR) preserve AsyncLocalStorage context in finished() (Gürgün Dayıoğlu) #57865 util: * (SEMVER-MINOR) add `types.isFloat16Array()` (Livia Medeiros) #57879 worker: * (SEMVER-MINOR) add worker.getHeapStatistics() (Matteo Collina) #57888 PR-URL: TODO

Notable changes: deps: * update timezone to 2025b (Node.js GitHub Bot) #57857 doc: * add dario-piotrowicz to collaborators (Dario Piotrowicz) #58102 * (SEMVER-MINOR) graduate multiple experimental apis (James M Snell) #57765 esm: * (SEMVER-MINOR) graduate import.meta properties (James M Snell) #58011 * (SEMVER-MINOR) support top-level Wasm without package type (Guy Bedford) #57610 sqlite: * (SEMVER-MINOR) add StatementSync.prototype.columns() (Colin Ihrig) #57490 src: * (SEMVER-MINOR) set default config as `node.config.json` (Marco Ippolito) #57171 * (SEMVER-MINOR) create `THROW_ERR_OPTIONS_BEFORE_BOOTSTRAPPING` (Marco Ippolito) #57016 * (SEMVER-MINOR) add config file support (Marco Ippolito) #57016 * (SEMVER-MINOR) add ExecutionAsyncId getter for any Context (Attila Szegedi) #57820 stream: * (SEMVER-MINOR) preserve AsyncLocalStorage context in finished() (Gürgün Dayıoğlu) #57865 util: * (SEMVER-MINOR) add `types.isFloat16Array()` (Livia Medeiros) #57879 worker: * (SEMVER-MINOR) add worker.getHeapStatistics() (Matteo Collina) #57888 PR-URL: #58388

Notable changes: deps: * update timezone to 2025b (Node.js GitHub Bot) nodejs#57857 doc: * add dario-piotrowicz to collaborators (Dario Piotrowicz) nodejs#58102 * (SEMVER-MINOR) graduate multiple experimental apis (James M Snell) nodejs#57765 esm: * (SEMVER-MINOR) graduate import.meta properties (James M Snell) nodejs#58011 * (SEMVER-MINOR) support top-level Wasm without package type (Guy Bedford) nodejs#57610 sqlite: * (SEMVER-MINOR) add StatementSync.prototype.columns() (Colin Ihrig) nodejs#57490 src: * (SEMVER-MINOR) set default config as `node.config.json` (Marco Ippolito) nodejs#57171 * (SEMVER-MINOR) create `THROW_ERR_OPTIONS_BEFORE_BOOTSTRAPPING` (Marco Ippolito) nodejs#57016 * (SEMVER-MINOR) add config file support (Marco Ippolito) nodejs#57016 * (SEMVER-MINOR) add ExecutionAsyncId getter for any Context (Attila Szegedi) nodejs#57820 stream: * (SEMVER-MINOR) preserve AsyncLocalStorage context in finished() (Gürgün Dayıoğlu) nodejs#57865 util: * (SEMVER-MINOR) add `types.isFloat16Array()` (Livia Medeiros) nodejs#57879 worker: * (SEMVER-MINOR) add worker.getHeapStatistics() (Matteo Collina) nodejs#57888 PR-URL: nodejs#58388

Notable changes: deps: * update timezone to 2025b (Node.js GitHub Bot) #57857 doc: * add dario-piotrowicz to collaborators (Dario Piotrowicz) #58102 * (SEMVER-MINOR) graduate multiple experimental apis (James M Snell) #57765 esm: * (SEMVER-MINOR) graduate import.meta properties (James M Snell) #58011 * (SEMVER-MINOR) support top-level Wasm without package type (Guy Bedford) #57610 sqlite: * (SEMVER-MINOR) add StatementSync.prototype.columns() (Colin Ihrig) #57490 src: * (SEMVER-MINOR) set default config as `node.config.json` (Marco Ippolito) #57171 * (SEMVER-MINOR) create `THROW_ERR_OPTIONS_BEFORE_BOOTSTRAPPING` (Marco Ippolito) #57016 * (SEMVER-MINOR) add config file support (Marco Ippolito) #57016 * (SEMVER-MINOR) add ExecutionAsyncId getter for any Context (Attila Szegedi) #57820 stream: * (SEMVER-MINOR) preserve AsyncLocalStorage context in finished() (Gürgün Dayıoğlu) #57865 util: * (SEMVER-MINOR) add `types.isFloat16Array()` (Livia Medeiros) #57879 worker: * (SEMVER-MINOR) add worker.getHeapStatistics() (Matteo Collina) #57888 PR-URL: #58388

mcollina requested review from anonrig, jasnell, joyeecheung and legendecas April 15, 2025 10:28

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Apr 15, 2025

mcollina added the semver-minor PRs that contain new features and should be released in the next minor version. label Apr 15, 2025

mcollina force-pushed the worker-heap-statistics branch from daafec6 to 85d6960 Compare April 15, 2025 10:36

mcollina added 3 commits April 15, 2025 12:58

worker: add worker.getHeapStatistics()

94ec4e6

Adds worker.getHeapStatistics() so that the heap usage of the worker could be observer from the parent thread. Signed-off-by: Matteo Collina <[email protected]>

fixup

852078c

Signed-off-by: Matteo Collina <[email protected]>

fixup

c73d1b6

Signed-off-by: Matteo Collina <[email protected]>

mcollina force-pushed the worker-heap-statistics branch from c10f5e8 to c73d1b6 Compare April 15, 2025 10:59

anonrig reviewed Apr 15, 2025

View reviewed changes

mcollina added 3 commits April 15, 2025 13:05

fixup

599fb2c

Signed-off-by: Matteo Collina <[email protected]>

fixup

d219327

Signed-off-by: Matteo Collina <[email protected]>

fixup

8b1266a

Signed-off-by: Matteo Collina <[email protected]>

Qard reviewed Apr 15, 2025

View reviewed changes

legendecas reviewed Apr 15, 2025

View reviewed changes

src/node_worker.cc Outdated Show resolved Hide resolved

RaisinTen reviewed Apr 15, 2025

View reviewed changes

src/node_worker.cc Outdated Show resolved Hide resolved

mcollina added 3 commits April 15, 2025 18:58

macros

09741f2

Signed-off-by: Matteo Collina <[email protected]>

object initialization

76d4027

Signed-off-by: Matteo Collina <[email protected]>

refactor to use an interrupt

47bd934

Signed-off-by: Matteo Collina <[email protected]>

mcollina added 2 commits April 15, 2025 19:43

fixup

ea24ef4

Signed-off-by: Matteo Collina <[email protected]>

fixup

821744f

Signed-off-by: Matteo Collina <[email protected]>

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 16, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 16, 2025

mcollina added the commit-queue Add this label to land a pull request using GitHub Actions. label Apr 17, 2025

RaisinTen added the commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. label Apr 17, 2025

nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Apr 17, 2025

nodejs-github-bot merged commit 33d8e03 into nodejs:main Apr 17, 2025
63 checks passed

mcollina deleted the worker-heap-statistics branch April 17, 2025 13:43

mcollina mentioned this pull request Apr 18, 2025

2025-05-06, Version 24.0.0 (Current) #57609

Merged

github-actions bot mentioned this pull request May 19, 2025

2025-05-21, Version 22.16.0 'Jod' (LTS) #58388

Merged

ghost mentioned this pull request Jun 8, 2025

### Notable Changes autofix-ci/action#28

Closed


		v8::HeapStatistics heap_stats;
		w->isolate_->GetHeapStatistics(&heap_stats);

Uh oh!

Conversation

mcollina commented Apr 15, 2025

Uh oh!

mcollina commented Apr 15, 2025

Uh oh!

anonrig Apr 15, 2025

Choose a reason for hiding this comment

Uh oh!

mcollina Apr 15, 2025

Choose a reason for hiding this comment

Uh oh!

anonrig Apr 15, 2025

Choose a reason for hiding this comment

Uh oh!

Qard left a comment

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Apr 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

legendecas commented Apr 15, 2025

Uh oh!

mcollina commented Apr 15, 2025

Uh oh!

Uh oh!

mcollina commented Apr 15, 2025

Uh oh!

nodejs-github-bot commented Apr 16, 2025

Uh oh!

nodejs-github-bot commented Apr 17, 2025

Uh oh!

Uh oh!

nodejs-github-bot commented Apr 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

codecov bot commented Apr 15, 2025 •

edited

Loading