crypto: reduce range of size to int max

[Ayase-252]

Background

A bump of max length of Buffer to 2 ** 32 - 1(https://bugs.chromium.org/p/v8/issues/detail?id=4153#c66) breaks validation of sizeargument of randomBytes. When passing a over-large size like 2 * 31), it causes abort in v14 as described in #38090.

Impact of the PR

For v15, the overlarge size is catched in C++ code, and an Error will throw currently:

Welcome to Node.js v15.12.0.
Type ".help" for more information.
> crypto.randomBytes(2147483648)
Uncaught RangeError: buffer is too large
    at randomFillSync (node:internal/crypto/random:128:15)
    at Object.randomBytes (node:internal/crypto/random:96:5) {
  code: 'ERR_OUT_OF_RANGE'

After the PR, an JS-level Error will throw:

> crypto.randomBytes(2147483648)
Uncaught:
RangeError [ERR_OUT_OF_RANGE]: The value of "size" is out of range. It must be >= 0 && <= 2147483647. Received 2147483648
    at new NodeError (node:internal/errors:349:5)
    at assertSize (node:internal/crypto/random:80:11)
    at Object.randomBytes (node:internal/crypto/random:93:10)
    at REPL1:1:8
    at Script.runInThisContext (node:vm:131:12)
    at REPLServer.defaultEval (node:repl:522:29)
    at bound (node:domain:416:15)
    at REPLServer.runBound [as eval] (node:domain:427:12)
    at REPLServer.onLine (node:repl:844:10)
    at REPLServer.emit (node:events:381:22) {
  code: 'ERR_OUT_OF_RANGE'
}

For v14, executing crypto.randomBytes(2147483648) will abort immediately

Welcome to Node.js v14.16.0.
Type ".help" for more information.
>  crypto.randomBytes(2147483648)
[1]    96221 segmentation fault  node

If this PR is backported successfully, it will throw the same Error described above instead of abort.

Refs: #38090

[RaisinTen]

Should we rather allow buffer sizes as large as 2 ** 32 - 1 (for 64-bit platforms)?

[RaisinTen]

A bump of max length of Buffer to 2 ** 32

@Ayase-252 Did you mean this in the PR description?

- A bump of max length of Buffer to 2 ** 32 
+ A bump of max length of Buffer to 2 ** 32 - 1

[Ayase-252]

A bump of max length of Buffer to 2 ** 32

@Ayase-252 Did you mean this in the PR description?

- A bump of max length of Buffer to 2 ** 32 
+ A bump of max length of Buffer to 2 ** 32 - 1

Yes, my mistake, I will edit in the OP too

[Ayase-252]

Should we rather allow buffer sizes as large as 2 ** 32 - 1 (for 64-bit platforms)?

Yes. It could be. But I don’t know find how crypto.random allocates buffer and why 2**31 - 1 is a limitation factor so far. May investigate tomorrow.

[Linkgoron]

@Ayase-252 I think you referenced the wrong issue in the commit message.

[Ayase-252]

@Ayase-252 I think you referenced the wrong issue in the commit message.

Thanks! Commit message is amended.

[jasnell]

Should we rather allow buffer sizes as large as 2 ** 32 - 1 (for 64-bit platforms)?

For now, no. Openssl is still pretty limited here, using int for many sizes. Enforcing the current limit is the right thing to do here.

[RaisinTen]

Instead of Refs, should we use Fixes in the commit message, as this stops the segfault?

[Ayase-252]

@RaisinTen I don't think this PR fixes #38090 until change is backported to v14 later. May it be more approriate to use Fixes in a backport commit?

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/37412/

[jasnell]

Landed in 993ed19