Skip to content

src: add option to report large page stats#32331

Closed
gabrielschulhof wants to merge 1 commit intonodejs:masterfrom
gabrielschulhof:large-pages-verbose
Closed

src: add option to report large page stats#32331
gabrielschulhof wants to merge 1 commit intonodejs:masterfrom
gabrielschulhof:large-pages-verbose

Conversation

@gabrielschulhof
Copy link
Contributor

@gabrielschulhof gabrielschulhof commented Mar 17, 2020

This adds the new option verbose to --use-largepages that works the
same way as on, but additionally outputs memory range and page count
information to stderr.

Signed-off-by: @gabrielschulhof

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • documentation is changed or added
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added the c++ Issues and PRs that require attention from people who are familiar with C++. label Mar 17, 2020
@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Mar 18, 2020

CI: https://ci.nodejs.org/job/node-test-pull-request/29906/

@bnoordhuis
Copy link
Member

bnoordhuis commented Mar 18, 2020

Before I review this: when or why (or to whom) would this be useful?

@cjihrig
Copy link
Contributor

cjihrig commented Mar 18, 2020 •
edited
Loading

Is this something that could leverage the existing NODE_DEBUG_NATIVE environment variable, instead of adding a new mode?

@gabrielschulhof
Copy link
Contributor Author

gabrielschulhof commented Mar 18, 2020 •
edited
Loading

@cjihrig I'd love to switch to NODE_DEBUG_NATIVE but, according to the documentation, its value is a list of native modules to debug, whereas the large pages statistics do not originate from any built-in native module, but rather from the runtime itself.

Is it within the semantics of NODE_DEBUG_NATIVE that I should could fprintf(stderr, ... if NODE_DEBUG_NATIVE is set to any non-NULL value?

@gabrielschulhof
Copy link
Contributor Author

gabrielschulhof commented Mar 18, 2020

@bnoordhuis I've used these fprintfs to see if I can maximize the code getting mapped to large pages. Another use might be if you see no benefit from running with --use-largepages=on. Then you might run with --use-largepages=verbose and see if the code that is hot for you is getting snipped off because it is at the beginning or end of the range we remap.

@cjihrig
Copy link
Contributor

cjihrig commented Mar 18, 2020

according to the documentation, its value is a list of native modules to debug, whereas the large pages statistics do not originate from any built-in native module, but rather from the runtime itself.

I think that definition might only be loosely enforced in the code. I'd personally be fine with a large pages category (and would prefer it over this approach). I guess we could see if anyone else objects though.

Is it within the semantics of NODE_DEBUG_NATIVE that I should fprintf(stderr, ... if NODE_DEBUG_NATIVE is set to any non-NULL value?

I don't think you'd use fprintf() explicitly. There are helpers in place already. This is how it's done in WASI, for example.

@gabrielschulhof
Copy link
Contributor Author

gabrielschulhof commented Mar 18, 2020

@cjihrig thanks for the pointer!

@jasnell
Copy link
Member

jasnell commented Mar 18, 2020

I would definitely prefer just using NODE_DEBUG_NATIVE for this. You can easily add a new category specific for this and it doesn't have to be limited to just debug output for modules.

@gabrielschulhof
Copy link
Contributor Author

gabrielschulhof commented Mar 19, 2020

@jasnell @cjihrig I looked at the implementation of Debug, and it requires an Environment*. The problem is, when this code runs, the Environment* is not yet created. So, I would have to check NODE_DEBUG_NATIVE by hand.

@jasnell
Copy link
Member

jasnell commented Mar 19, 2020

hmm ok. Is it possible/reasonable to defer the logging until Environment is available?

@gabrielschulhof
Copy link
Contributor Author

gabrielschulhof commented Mar 19, 2020

@jasnell looks like there might be a way to use the debug without an Environment 🤔

src: add debug option to report large page stats
436cc4d 
This adds the new option `HUGEPAGES` to `NODE_DEBUG_NATIVE` that
causes the code responsible for re-mapping to large pages to output
memory range and page count information to `stderr`.

Signed-off-by: Gabriel Schulhof <gabriel.schulhof@intel.com>
@gabrielschulhof
Copy link
Contributor Author

gabrielschulhof commented Mar 20, 2020

@jasnell @cjihrig I updated the PR to use Debug().

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Mar 20, 2020

CI: https://ci.nodejs.org/job/node-test-pull-request/29955/

@gabrielschulhof
Copy link
Contributor Author

gabrielschulhof commented Mar 20, 2020

Landed in c160073.

gabrielschulhof pushed a commit that referenced this pull request Mar 20, 2020
src: add debug option to report large page stats
c160073 
This adds the new option `HUGEPAGES` to `NODE_DEBUG_NATIVE` that
causes the code responsible for re-mapping to large pages to output
memory range and page count information to `stderr`.

Signed-off-by: Gabriel Schulhof <gabriel.schulhof@intel.com>
PR-URL: #32331
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
@gabrielschulhof gabrielschulhof deleted the large-pages-verbose branch March 20, 2020 17:45
MylesBorins pushed a commit that referenced this pull request Mar 24, 2020
@MylesBorins
src: add debug option to report large page stats
243d0d4 
This adds the new option `HUGEPAGES` to `NODE_DEBUG_NATIVE` that
causes the code responsible for re-mapping to large pages to output
memory range and page count information to `stderr`.

Signed-off-by: Gabriel Schulhof <gabriel.schulhof@intel.com>
PR-URL: #32331
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
@MylesBorins MylesBorins mentioned this pull request Mar 24, 2020
Merged
@targos
Copy link
Member

targos commented Apr 22, 2020

Depends on the large pages change to land on v12.x

targos pushed a commit to targos/node that referenced this pull request Apr 25, 2020
@targos
src: add debug option to report large page stats
193d634 
This adds the new option `HUGEPAGES` to `NODE_DEBUG_NATIVE` that
causes the code responsible for re-mapping to large pages to output
memory range and page count information to `stderr`.

Signed-off-by: Gabriel Schulhof <gabriel.schulhof@intel.com>
PR-URL: nodejs#32331
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
targos pushed a commit that referenced this pull request Apr 28, 2020
@targos
src: add debug option to report large page stats
df046de 
This adds the new option `HUGEPAGES` to `NODE_DEBUG_NATIVE` that
causes the code responsible for re-mapping to large pages to output
memory range and page count information to `stderr`.

Signed-off-by: Gabriel Schulhof <gabriel.schulhof@intel.com>
PR-URL: #32331
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
@targos targos mentioned this pull request May 2, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bnoordhuis bnoordhuis bnoordhuis approved these changes

@cjihrig cjihrig cjihrig approved these changes

@lundibundi lundibundi Awaiting requested review from lundibundi

+1 more reviewer

@devnexen devnexen devnexen approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

c++ Issues and PRs that require attention from people who are familiar with C++.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@gabrielschulhof @nodejs-github-bot @bnoordhuis @cjihrig @jasnell @targos @devnexen