[v8.x backport] Backport tsfn to v8.x#25002
[v8.x backport] Backport tsfn to v8.x#25002gabrielschulhof wants to merge 10 commits intonodejs:v8.x-stagingfrom
Conversation
src/node_api.cc
Outdated
There was a problem hiding this comment.
Notable difference: node::Persistent is unavailable in v8.x, so ref is a normal v8::Persistent which requires resetting upon instance destruction.
src/node_api.cc
Outdated
There was a problem hiding this comment.
Notable difference: env->CloseHandle() is unavailable in v8.x so this uses plain uv_close().
src/node_api.cc
Outdated
There was a problem hiding this comment.
Similarly, using plain uv_close() here because there is no env->CloseHandle().
src/node_api.cc
Outdated
There was a problem hiding this comment.
Same here (wrt. using plain uv_close() because env->CloseHandle() is unavailable).
|
Re: #24249 |
|
This backport incorporates the following commits: 2b1dd6c test: mark test_threadsafe_function/test as flaky |
|
@gabrielschulhof is this functionality available on v6.x? |
c5142b3 to
3af7528
Compare
|
@MylesBorins I don't believe so. |
Bundle a `uv_async_t`, a `uv_idle_t`, a `uv_mutex_t`, a `uv_cond_t`, and a `v8::Persistent<v8::Function>` to make it possible to call into JS from another thread. The API accepts a void data pointer and a callback which will be invoked on the loop thread and which will receive the `napi_value` representing the JavaScript function to call so as to perform the call into JS. The callback is run inside a `node::CallbackScope`. A `std::queue<void*>` is used to store calls from the secondary threads, and an idle loop is started by the `uv_async_t` callback on the loop thread to drain the queue, calling into JS with each item. Items can be added to the queue blockingly or non-blockingly. The thread-safe function can be referenced or unreferenced, with the same semantics as libuv handles. Re: nodejs/help#1035 Re: nodejs#20964 Fixes: nodejs#13512 PR-URL: nodejs#17887 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
private field 'async_context' is not used [-Wunused-private-field] PR-URL: nodejs#21597 Refs: nodejs#17887 Reviewed-By: Gabriel Schulhof <gabriel.schulhof@intel.com> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
A condition variable is only created by the thread-safe function if the queue size is set to something larger than zero. This adds null-checks around the condition variable and tests for the case where the queue size is zero. Fixes: nodejs/help#1387 PR-URL: nodejs#21871 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
PR-URL: nodejs#21898 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com> Reviewed-By: Jon Moss <me@jonathanmoss.me> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
The idle_running member variable in TsFn is always false and can therefore be removed. PR-URL: nodejs#22520 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Gabriel Schulhof <gabriel.schulhof@intel.com> Reviewed-By: Gus Caplan <me@gus.host>
* Move class `TsFn` to name space `v8impl` and rename it to `ThreadSafeFunction` * Remove `NAPI_EXTERN` from API declarations, because it's only needed in the header file. PR-URL: nodejs#22259 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Kyle Farnung <kfarnung@microsoft.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
The thread_finalize_data and thread_finalize_cb parameters in napi_create_threadsafe_function are optional. PR-URL: nodejs#22998 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>
Currently when building with --debug
test/addons-napi/test_threadsafe_function will error:
$ out/Debug/node test/addons-napi/test_threadsafe_function/test.js
FATAL ERROR: v8::HandleScope::CreateHandle()
Cannot create a handle without a HandleScope
1: 0x10004e287 node::DumpBacktrace(__sFILE*) [node/out/Debug/node]
2: 0x1000cd37b node::Abort() [/node/out/Debug/node]
3: 0x1000cd69f node::OnFatalError(char const*, char const*)
[/node/out/Debug/node]
4: 0x1004df0b1 v8::Utils::ReportApiFailure(char const*, char const*)
[/nodejs/node/out/Debug/node]
5: 0x100a8c0a9 v8::internal::HandleScope::Extend(
v8::internal::Isolate*)
[/node/out/Debug/node]
6: 0x1004e4229 v8::EmbedderDataFor(v8::Context*,
int, bool,
char const*)
[/node/out/Debug/node]
7: 0x1004e43fa v8::Context::SlowGetAlignedPointerFromEmbedderData(int)
[/node/out/Debug/node]
8: 0x10001c26b v8::Context::GetAlignedPointerFromEmbedderData(int)
[/node/out/Debug/node]
9: 0x1000144ea node::Environment::GetCurrent(v8::Local<v8::Context>)
[/node/out/Debug/node]
10: 0x1000f49e2 napi_env__::node_env() const
[/node/out/Debug/node]
11: 0x1000f9885
(anonymous namespace)::v8impl::ThreadSafeFunction::
CloseHandlesAndMaybeDelete(bool)
[/node/out/Debug/node]
12: 0x1000fb34f (anonymous namespace)::v8impl::ThreadSafeFunction::
DispatchOne()
[/node/out/Debug/node]
13: 0x1000fb129
(anonymous namespace)::v8impl::ThreadSafeFunction::
IdleCb(uv_idle_s*)
[/node/out/Debug/node]
14: 0x1011a1b69 uv__run_idle
[/node/out/Debug/node]
15: 0x101198179 uv_run
[/node/out/Debug/node]
16: 0x1000dfca1
node::Start(...)
[/node/out/Debug/node]
17: 0x1000dae50 node::Start(...)
[/node/out/Debug/node]
18: 0x1000da56f node::Start(int, char**)
[/node/out/Debug/node]
19: 0x10141112e main
[/node/out/Debug/node]
20: 0x100001034 start
[/node/out/Debug/node]
Abort trap: 6
This commit adds two HandleScope's, one to CloseHandlesAndMaybeDelete
and one to the lambda.
SlowGetAlignedPointerFromEmbedderData will only be called for debug
builds:
https://github.com/v8/v8/blob/2ef0aa662fe907a1b36ac1abe7d77ad2bcd27733
/include/v8.h#L10440-L10447
PR-URL: nodejs#24011
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
The test fails consistently on windows-fanned with vs2017. mark it as flaky while the issue is being progressed, and to keep CI green / amber. Ref: nodejs#23621 PR-URL: nodejs#24714 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>
4f1a9fa to
e0e1ac2
Compare
|
Re-based. CI: https://ci.nodejs.org/job/node-test-pull-request/19859/ |
|
Some of these commits are We have also discussed backporting I've opened an issue to discuss this nodejs/Release#405 I've dismissed my blocking review, but would like us to reach a conclusion in the release issue before landing this. |
|
@gabrielschulhof I think the differences that you pointed out are all very much fine here 👍 |
|
This is the only thing preventing us from deprecating fsevents@1.x entirely in april. So PLEASE 👍 I beg you! |
|
@MylesBorins @gabrielschulhof ping please |
Bundle a `uv_async_t`, a `uv_idle_t`, a `uv_mutex_t`, a `uv_cond_t`, and a `v8::Persistent<v8::Function>` to make it possible to call into JS from another thread. The API accepts a void data pointer and a callback which will be invoked on the loop thread and which will receive the `napi_value` representing the JavaScript function to call so as to perform the call into JS. The callback is run inside a `node::CallbackScope`. A `std::queue<void*>` is used to store calls from the secondary threads, and an idle loop is started by the `uv_async_t` callback on the loop thread to drain the queue, calling into JS with each item. Items can be added to the queue blockingly or non-blockingly. The thread-safe function can be referenced or unreferenced, with the same semantics as libuv handles. Re: nodejs/help#1035 Re: #20964 Fixes: #13512 Backport-PR-URL: #25002 PR-URL: #17887 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
A condition variable is only created by the thread-safe function if the queue size is set to something larger than zero. This adds null-checks around the condition variable and tests for the case where the queue size is zero. Fixes: nodejs/help#1387 PR-URL: #21871 Backport-PR-URL: #25002 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Backport-PR-URL: #25002 PR-URL: #21898 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com> Reviewed-By: Jon Moss <me@jonathanmoss.me> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
The idle_running member variable in TsFn is always false and can therefore be removed. Backport-PR-URL: #25002 PR-URL: #22520 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Gabriel Schulhof <gabriel.schulhof@intel.com> Reviewed-By: Gus Caplan <me@gus.host>
* Move class `TsFn` to name space `v8impl` and rename it to `ThreadSafeFunction` * Remove `NAPI_EXTERN` from API declarations, because it's only needed in the header file. Backport-PR-URL: #25002 PR-URL: #22259 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Kyle Farnung <kfarnung@microsoft.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Currently when building with --debug
test/addons-napi/test_threadsafe_function will error:
$ out/Debug/node test/addons-napi/test_threadsafe_function/test.js
FATAL ERROR: v8::HandleScope::CreateHandle()
Cannot create a handle without a HandleScope
1: 0x10004e287 node::DumpBacktrace(__sFILE*) [node/out/Debug/node]
2: 0x1000cd37b node::Abort() [/node/out/Debug/node]
3: 0x1000cd69f node::OnFatalError(char const*, char const*)
[/node/out/Debug/node]
4: 0x1004df0b1 v8::Utils::ReportApiFailure(char const*, char const*)
[/nodejs/node/out/Debug/node]
5: 0x100a8c0a9 v8::internal::HandleScope::Extend(
v8::internal::Isolate*)
[/node/out/Debug/node]
6: 0x1004e4229 v8::EmbedderDataFor(v8::Context*,
int, bool,
char const*)
[/node/out/Debug/node]
7: 0x1004e43fa v8::Context::SlowGetAlignedPointerFromEmbedderData(int)
[/node/out/Debug/node]
8: 0x10001c26b v8::Context::GetAlignedPointerFromEmbedderData(int)
[/node/out/Debug/node]
9: 0x1000144ea node::Environment::GetCurrent(v8::Local<v8::Context>)
[/node/out/Debug/node]
10: 0x1000f49e2 napi_env__::node_env() const
[/node/out/Debug/node]
11: 0x1000f9885
(anonymous namespace)::v8impl::ThreadSafeFunction::
CloseHandlesAndMaybeDelete(bool)
[/node/out/Debug/node]
12: 0x1000fb34f (anonymous namespace)::v8impl::ThreadSafeFunction::
DispatchOne()
[/node/out/Debug/node]
13: 0x1000fb129
(anonymous namespace)::v8impl::ThreadSafeFunction::
IdleCb(uv_idle_s*)
[/node/out/Debug/node]
14: 0x1011a1b69 uv__run_idle
[/node/out/Debug/node]
15: 0x101198179 uv_run
[/node/out/Debug/node]
16: 0x1000dfca1
node::Start(...)
[/node/out/Debug/node]
17: 0x1000dae50 node::Start(...)
[/node/out/Debug/node]
18: 0x1000da56f node::Start(int, char**)
[/node/out/Debug/node]
19: 0x10141112e main
[/node/out/Debug/node]
20: 0x100001034 start
[/node/out/Debug/node]
Abort trap: 6
This commit adds two HandleScope's, one to CloseHandlesAndMaybeDelete
and one to the lambda.
SlowGetAlignedPointerFromEmbedderData will only be called for debug
builds:
https://github.com/v8/v8/blob/2ef0aa662fe907a1b36ac1abe7d77ad2bcd27733
/include/v8.h#L10440-L10447
Backport-PR-URL: #25002
PR-URL: #24011
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
|
landed in 48da8be...b44bd88 |
|
Huge. Thanks Myles! |
|
Awesome 🎉🎉🎉🎉 |
Bundle a `uv_async_t`, a `uv_idle_t`, a `uv_mutex_t`, a `uv_cond_t`, and a `v8::Persistent<v8::Function>` to make it possible to call into JS from another thread. The API accepts a void data pointer and a callback which will be invoked on the loop thread and which will receive the `napi_value` representing the JavaScript function to call so as to perform the call into JS. The callback is run inside a `node::CallbackScope`. A `std::queue<void*>` is used to store calls from the secondary threads, and an idle loop is started by the `uv_async_t` callback on the loop thread to drain the queue, calling into JS with each item. Items can be added to the queue blockingly or non-blockingly. The thread-safe function can be referenced or unreferenced, with the same semantics as libuv handles. Re: nodejs/help#1035 Re: #20964 Fixes: #13512 Backport-PR-URL: #25002 PR-URL: #17887 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
A condition variable is only created by the thread-safe function if the queue size is set to something larger than zero. This adds null-checks around the condition variable and tests for the case where the queue size is zero. Fixes: nodejs/help#1387 PR-URL: #21871 Backport-PR-URL: #25002 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Backport-PR-URL: #25002 PR-URL: #21898 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com> Reviewed-By: Jon Moss <me@jonathanmoss.me> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
The idle_running member variable in TsFn is always false and can therefore be removed. Backport-PR-URL: #25002 PR-URL: #22520 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Gabriel Schulhof <gabriel.schulhof@intel.com> Reviewed-By: Gus Caplan <me@gus.host>
* Move class `TsFn` to name space `v8impl` and rename it to `ThreadSafeFunction` * Remove `NAPI_EXTERN` from API declarations, because it's only needed in the header file. Backport-PR-URL: #25002 PR-URL: #22259 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Kyle Farnung <kfarnung@microsoft.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Currently when building with --debug
test/addons-napi/test_threadsafe_function will error:
$ out/Debug/node test/addons-napi/test_threadsafe_function/test.js
FATAL ERROR: v8::HandleScope::CreateHandle()
Cannot create a handle without a HandleScope
1: 0x10004e287 node::DumpBacktrace(__sFILE*) [node/out/Debug/node]
2: 0x1000cd37b node::Abort() [/node/out/Debug/node]
3: 0x1000cd69f node::OnFatalError(char const*, char const*)
[/node/out/Debug/node]
4: 0x1004df0b1 v8::Utils::ReportApiFailure(char const*, char const*)
[/nodejs/node/out/Debug/node]
5: 0x100a8c0a9 v8::internal::HandleScope::Extend(
v8::internal::Isolate*)
[/node/out/Debug/node]
6: 0x1004e4229 v8::EmbedderDataFor(v8::Context*,
int, bool,
char const*)
[/node/out/Debug/node]
7: 0x1004e43fa v8::Context::SlowGetAlignedPointerFromEmbedderData(int)
[/node/out/Debug/node]
8: 0x10001c26b v8::Context::GetAlignedPointerFromEmbedderData(int)
[/node/out/Debug/node]
9: 0x1000144ea node::Environment::GetCurrent(v8::Local<v8::Context>)
[/node/out/Debug/node]
10: 0x1000f49e2 napi_env__::node_env() const
[/node/out/Debug/node]
11: 0x1000f9885
(anonymous namespace)::v8impl::ThreadSafeFunction::
CloseHandlesAndMaybeDelete(bool)
[/node/out/Debug/node]
12: 0x1000fb34f (anonymous namespace)::v8impl::ThreadSafeFunction::
DispatchOne()
[/node/out/Debug/node]
13: 0x1000fb129
(anonymous namespace)::v8impl::ThreadSafeFunction::
IdleCb(uv_idle_s*)
[/node/out/Debug/node]
14: 0x1011a1b69 uv__run_idle
[/node/out/Debug/node]
15: 0x101198179 uv_run
[/node/out/Debug/node]
16: 0x1000dfca1
node::Start(...)
[/node/out/Debug/node]
17: 0x1000dae50 node::Start(...)
[/node/out/Debug/node]
18: 0x1000da56f node::Start(int, char**)
[/node/out/Debug/node]
19: 0x10141112e main
[/node/out/Debug/node]
20: 0x100001034 start
[/node/out/Debug/node]
Abort trap: 6
This commit adds two HandleScope's, one to CloseHandlesAndMaybeDelete
and one to the lambda.
SlowGetAlignedPointerFromEmbedderData will only be called for debug
builds:
https://github.com/v8/v8/blob/2ef0aa662fe907a1b36ac1abe7d77ad2bcd27733
/include/v8.h#L10440-L10447
Backport-PR-URL: #25002
PR-URL: #24011
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Checklist
make -j4 test(UNIX), orvcbuild test(Windows) passesBackporting
ThreadSafeFunctionto v8.x completes its availability across all LTS release lines. This will, in turn, allow us to mark it as stable and thereby bump the available N-API version to 4.