fs: allow correct handling of burst in fs-events with AsyncIterator#58490
Merged
nodejs-github-bot merged 1 commit intonodejs:mainfrom Jun 10, 2025
Merged
fs: allow correct handling of burst in fs-events with AsyncIterator#58490nodejs-github-bot merged 1 commit intonodejs:mainfrom
nodejs-github-bot merged 1 commit intonodejs:mainfrom
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #58490 +/- ##
==========================================
- Coverage 90.16% 90.15% -0.01%
==========================================
Files 636 636
Lines 187891 187923 +32
Branches 36884 36882 -2
==========================================
+ Hits 169408 169429 +21
- Misses 11246 11247 +1
- Partials 7237 7247 +10
🚀 New features to boost your workflow:
|
63d31e7 to
3512480
Compare
joyeecheung
reviewed
May 29, 2025
joyeecheung
reviewed
May 30, 2025
16c2f33 to
d5d6d5d
Compare
joyeecheung
reviewed
Jun 4, 2025
joyeecheung
reviewed
Jun 6, 2025
Member
|
This almost LGTM % the suggestions. Though I'd appreciate if others from @nodejs/fs or maybe @jasnell @benjamingr can take a look and see if the new options look good. |
Contributor
Ethan-Arrowood
left a comment
There was a problem hiding this comment.
Other than the option values, this LGTM
joyeecheung
reviewed
Jun 6, 2025
joyeecheung
approved these changes
Jun 6, 2025
Member
joyeecheung
left a comment
There was a problem hiding this comment.
LGTM % missing updates to the docs
Contributor
Author
|
Thanks for all the help, guidance & input @joyeecheung & @Ethan-Arrowood ❤️ |
Ethan-Arrowood
approved these changes
Jun 6, 2025
joyeecheung
approved these changes
Jun 6, 2025
Collaborator
Collaborator
Collaborator
|
Landed in 5f7dbf4 |
seriousme
pushed a commit
to seriousme/node
that referenced
this pull request
Jun 10, 2025
PR-URL: nodejs#58490 Reviewed-By: Joyee Cheung <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]>
targos
pushed a commit
that referenced
this pull request
Jun 16, 2025
PR-URL: #58490 Reviewed-By: Joyee Cheung <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]>
nodejs-github-bot
added a commit
that referenced
this pull request
Jun 16, 2025
Notable changes: doc: * add islandryu to collaborators (Shima Ryuhei) #58714 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #58643 test_runner: * (SEMVER-MINOR) support object property mocking (Idan Goshen) #58438 url: * (SEMVER-MINOR) add fileURLToPathBuffer API (James M Snell) #58700 PR-URL: #58727
nodejs-github-bot
added a commit
that referenced
this pull request
Jun 23, 2025
Notable changes: doc: * add islandryu to collaborators (Shima Ryuhei) #58714 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #58643 test_runner: * (SEMVER-MINOR) support object property mocking (Idan Goshen) #58438 url: * (SEMVER-MINOR) add fileURLToPathBuffer API (James M Snell) #58700 PR-URL: #58804
nodejs-github-bot
added a commit
that referenced
this pull request
Jun 24, 2025
Notable changes: doc: * add islandryu to collaborators (Shima Ryuhei) #58714 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #58643 test: * fix test-timeout-flag after revert of auto subtest wait (Pietro Marchini) #58282 test_runner: * Revert "test_runner: remove promises returned by t.test() (Romain Menke) #58282 * Revert "test_runner: remove promises returned by test() (Romain Menke) #58282 * Revert "test_runner: automatically wait for subtests to finish (Romain Menke) #58282 * (SEMVER-MINOR) support object property mocking (Idan Goshen) #58438 url: * (SEMVER-MINOR) add fileURLToPathBuffer API (James M Snell) #58700 PR-URL: #58813
RafaelGSS
added a commit
that referenced
this pull request
Jun 24, 2025
Notable changes: doc: * add islandryu to collaborators (Shima Ryuhei) #58714 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #58643 test: * fix test-timeout-flag after revert of auto subtest wait (Pietro Marchini) #58282 test_runner: * (SEMVER-MINOR) support object property mocking (Idan Goshen) #58438 url: * (SEMVER-MINOR) add fileURLToPathBuffer API (James M Snell) #58700 PR-URL: #58813
RafaelGSS
added a commit
that referenced
this pull request
Jun 24, 2025
Notable changes: doc: * add islandryu to collaborators (Shima Ryuhei) #58714 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #58643 test: * fix test-timeout-flag after revert of auto subtest wait (Pietro Marchini) #58282 test_runner: * (SEMVER-MINOR) support object property mocking (Idan Goshen) #58438 url: * (SEMVER-MINOR) add fileURLToPathBuffer API (James M Snell) #58700 PR-URL: #58813
aduh95
pushed a commit
that referenced
this pull request
Jul 21, 2025
PR-URL: #58490 Reviewed-By: Joyee Cheung <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]>
aduh95
pushed a commit
that referenced
this pull request
Jul 24, 2025
PR-URL: #58490 Reviewed-By: Joyee Cheung <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]>
nodejs-github-bot
added a commit
that referenced
this pull request
Jul 28, 2025
Notable changes: deps: * (SEMVER-MINOR) update amaro to 1.1.0 (Node.js GitHub Bot) #56350 * (SEMVER-MINOR) update amaro to 1.0.0 (Node.js GitHub Bot) #56350 * (SEMVER-MINOR) update amaro to 0.5.3 (Node.js GitHub Bot) #56350 * (SEMVER-MINOR) update amaro to 0.5.2 (Node.js GitHub Bot) #56350 * (SEMVER-MINOR) update amaro to 0.5.1 (Marco Ippolito) #56350 * (SEMVER-MINOR) update amaro to 0.5.0 (nodejs-github-bot) #56350 doc: * (SEMVER-MINOR) add all watch-mode related flags to node.1 (Dario Piotrowicz) #58719 * add islandryu to collaborators (Shima Ryuhei) #58714 * (SEMVER-MINOR) add history entries to `--input-type` section (Antoine du Hamel) #56350 esm: * (SEMVER-MINOR) implement import.meta.main (Joe) #57804 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) improve typescript error message format (Marco Ippolito) #56350 * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #56350 * (SEMVER-MINOR) refactor commonjs typescript loader (Marco Ippolito) #56350 * (SEMVER-MINOR) unflag --experimental-strip-types (Marco Ippolito) #56350 permission: * (SEMVER-MINOR) propagate permission model flags on spawn (Rafael Gonzaga) #58853 sqlite: * (SEMVER-MINOR) add support for readBigInts option in db connection level (Miguel Marcondes Filho) #58697 src,permission: * (SEMVER-MINOR) add support to permission.has(addon) (Rafael Gonzaga) #58951 test: * (SEMVER-MINOR) add test for async disposable worker thread (James M Snell) #58385 url: * (SEMVER-MINOR) add fileURLToPathBuffer API (James M Snell) #58700 watch: * (SEMVER-MINOR) add `--watch-kill-signal` flag (Dario Piotrowicz) #58719 worker: * (SEMVER-MINOR) make Worker async disposable (James M Snell) #58385 PR-URL: #59256
aduh95
added a commit
that referenced
this pull request
Jul 28, 2025
Notable changes: deps: * (SEMVER-MINOR) update amaro to 1.1.0 (Node.js GitHub Bot) #56350 doc: * add islandryu to collaborators (Shima Ryuhei) #58714 esm: * (SEMVER-MINOR) implement `import.meta.main` (Joe) #57804 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #56350 * (SEMVER-MINOR) unflag `--experimental-strip-types` (Marco Ippolito) #56350 permission: * (SEMVER-MINOR) propagate permission model flags on spawn (Rafael Gonzaga) #58853 sqlite: * (SEMVER-MINOR) add support for `readBigInts` option in db connection level (Miguel Marcondes Filho) #58697 src,permission: * (SEMVER-MINOR) add support to `permission.has(addon)` (Rafael Gonzaga) #58951 url: * (SEMVER-MINOR) add `fileURLToPathBuffer` API (James M Snell) #58700 watch: * (SEMVER-MINOR) add `--watch-kill-signal` flag (Dario Piotrowicz) #58719 worker: * (SEMVER-MINOR) make `Worker` async disposable (James M Snell) #58385 PR-URL: #59256 Co-authored-by: Antoine du Hamel <[email protected]>
aduh95
added a commit
that referenced
this pull request
Jul 31, 2025
Notable changes: deps: * (SEMVER-MINOR) update amaro to 1.1.0 (Node.js GitHub Bot) #56350 doc: * add islandryu to collaborators (Shima Ryuhei) #58714 esm: * (SEMVER-MINOR) implement `import.meta.main` (Joe) #57804 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #56350 * (SEMVER-MINOR) unflag `--experimental-strip-types` (Marco Ippolito) #56350 permission: * (SEMVER-MINOR) propagate permission model flags on spawn (Rafael Gonzaga) #58853 sqlite: * (SEMVER-MINOR) add support for `readBigInts` option in db connection level (Miguel Marcondes Filho) #58697 src,permission: * (SEMVER-MINOR) add support to `permission.has(addon)` (Rafael Gonzaga) #58951 url: * (SEMVER-MINOR) add `fileURLToPathBuffer` API (James M Snell) #58700 watch: * (SEMVER-MINOR) add `--watch-kill-signal` flag (Dario Piotrowicz) #58719 worker: * (SEMVER-MINOR) make `Worker` async disposable (James M Snell) #58385 PR-URL: #59256 Co-authored-by: Antoine du Hamel <[email protected]>
aduh95
added a commit
that referenced
this pull request
Jul 31, 2025
Notable changes: deps: * (SEMVER-MINOR) update amaro to 1.1.0 (Node.js GitHub Bot) #56350 doc: * add islandryu to collaborators (Shima Ryuhei) #58714 esm: * (SEMVER-MINOR) implement `import.meta.main` (Joe) #57804 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #56350 * (SEMVER-MINOR) unflag `--experimental-strip-types` (Marco Ippolito) #56350 permission: * (SEMVER-MINOR) propagate permission model flags on spawn (Rafael Gonzaga) #58853 sqlite: * (SEMVER-MINOR) add support for `readBigInts` option in db connection level (Miguel Marcondes Filho) #58697 src,permission: * (SEMVER-MINOR) add support to `permission.has(addon)` (Rafael Gonzaga) #58951 url: * (SEMVER-MINOR) add `fileURLToPathBuffer` API (James M Snell) #58700 watch: * (SEMVER-MINOR) add `--watch-kill-signal` flag (Dario Piotrowicz) #58719 worker: * (SEMVER-MINOR) make `Worker` async disposable (James M Snell) #58385 PR-URL: #59256 Co-authored-by: Antoine du Hamel <[email protected]>
meteorqz6
pushed a commit
to meteorqz6/node
that referenced
this pull request
Aug 2, 2025
Notable changes: deps: * (SEMVER-MINOR) update amaro to 1.1.0 (Node.js GitHub Bot) nodejs#56350 doc: * add islandryu to collaborators (Shima Ryuhei) nodejs#58714 esm: * (SEMVER-MINOR) implement `import.meta.main` (Joe) nodejs#57804 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) nodejs#58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) nodejs#56350 * (SEMVER-MINOR) unflag `--experimental-strip-types` (Marco Ippolito) nodejs#56350 permission: * (SEMVER-MINOR) propagate permission model flags on spawn (Rafael Gonzaga) nodejs#58853 sqlite: * (SEMVER-MINOR) add support for `readBigInts` option in db connection level (Miguel Marcondes Filho) nodejs#58697 src,permission: * (SEMVER-MINOR) add support to `permission.has(addon)` (Rafael Gonzaga) nodejs#58951 url: * (SEMVER-MINOR) add `fileURLToPathBuffer` API (James M Snell) nodejs#58700 watch: * (SEMVER-MINOR) add `--watch-kill-signal` flag (Dario Piotrowicz) nodejs#58719 worker: * (SEMVER-MINOR) make `Worker` async disposable (James M Snell) nodejs#58385 PR-URL: nodejs#59256 Co-authored-by: Antoine du Hamel <[email protected]>
panva
pushed a commit
to panva/node
that referenced
this pull request
Aug 7, 2025
Notable changes: deps: * (SEMVER-MINOR) update amaro to 1.1.0 (Node.js GitHub Bot) nodejs#56350 doc: * add islandryu to collaborators (Shima Ryuhei) nodejs#58714 esm: * (SEMVER-MINOR) implement `import.meta.main` (Joe) nodejs#57804 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) nodejs#58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) nodejs#56350 * (SEMVER-MINOR) unflag `--experimental-strip-types` (Marco Ippolito) nodejs#56350 permission: * (SEMVER-MINOR) propagate permission model flags on spawn (Rafael Gonzaga) nodejs#58853 sqlite: * (SEMVER-MINOR) add support for `readBigInts` option in db connection level (Miguel Marcondes Filho) nodejs#58697 src,permission: * (SEMVER-MINOR) add support to `permission.has(addon)` (Rafael Gonzaga) nodejs#58951 url: * (SEMVER-MINOR) add `fileURLToPathBuffer` API (James M Snell) nodejs#58700 watch: * (SEMVER-MINOR) add `--watch-kill-signal` flag (Dario Piotrowicz) nodejs#58719 worker: * (SEMVER-MINOR) make `Worker` async disposable (James M Snell) nodejs#58385 PR-URL: nodejs#59256 Co-authored-by: Antoine du Hamel <[email protected]>
mete0rfish
pushed a commit
to mete0rfish/node-contribute
that referenced
this pull request
Aug 9, 2025
Notable changes: deps: * (SEMVER-MINOR) update amaro to 1.1.0 (Node.js GitHub Bot) nodejs#56350 doc: * add islandryu to collaborators (Shima Ryuhei) nodejs#58714 esm: * (SEMVER-MINOR) implement `import.meta.main` (Joe) nodejs#57804 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) nodejs#58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) nodejs#56350 * (SEMVER-MINOR) unflag `--experimental-strip-types` (Marco Ippolito) nodejs#56350 permission: * (SEMVER-MINOR) propagate permission model flags on spawn (Rafael Gonzaga) nodejs#58853 sqlite: * (SEMVER-MINOR) add support for `readBigInts` option in db connection level (Miguel Marcondes Filho) nodejs#58697 src,permission: * (SEMVER-MINOR) add support to `permission.has(addon)` (Rafael Gonzaga) nodejs#58951 url: * (SEMVER-MINOR) add `fileURLToPathBuffer` API (James M Snell) nodejs#58700 watch: * (SEMVER-MINOR) add `--watch-kill-signal` flag (Dario Piotrowicz) nodejs#58719 worker: * (SEMVER-MINOR) make `Worker` async disposable (James M Snell) nodejs#58385 PR-URL: nodejs#59256 Co-authored-by: Antoine du Hamel <[email protected]>
panva
pushed a commit
to panva/node
that referenced
this pull request
Aug 9, 2025
Notable changes: deps: * (SEMVER-MINOR) update amaro to 1.1.0 (Node.js GitHub Bot) nodejs#56350 doc: * add islandryu to collaborators (Shima Ryuhei) nodejs#58714 esm: * (SEMVER-MINOR) implement `import.meta.main` (Joe) nodejs#57804 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) nodejs#58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) nodejs#56350 * (SEMVER-MINOR) unflag `--experimental-strip-types` (Marco Ippolito) nodejs#56350 permission: * (SEMVER-MINOR) propagate permission model flags on spawn (Rafael Gonzaga) nodejs#58853 sqlite: * (SEMVER-MINOR) add support for `readBigInts` option in db connection level (Miguel Marcondes Filho) nodejs#58697 src,permission: * (SEMVER-MINOR) add support to `permission.has(addon)` (Rafael Gonzaga) nodejs#58951 url: * (SEMVER-MINOR) add `fileURLToPathBuffer` API (James M Snell) nodejs#58700 watch: * (SEMVER-MINOR) add `--watch-kill-signal` flag (Dario Piotrowicz) nodejs#58719 worker: * (SEMVER-MINOR) make `Worker` async disposable (James M Snell) nodejs#58385 PR-URL: nodejs#59256 Co-authored-by: Antoine du Hamel <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This addresses a bug in
fs.watchwhen used as an AsyncIterator.The issue is that when consuming the AsyncIteractor returned by
fs.watchit yields a value. When using that value and in turn awaiting an asynchrounous action any events happening in the meantime will go missing. The reason is that between exiting the watch function by yielding and reentering it through the next round, thepromiseinside watch is already resolved. So any events generated will be duplicate resolutions of that promise and therefore ignored.More reaslistically than this minimal example is when the found files are actually read via
await fs.readFile. Then there will be a lag. If the file events happening are happening close together, then the second one will be missed.To fix this issue I added a queue to the watch function that new file events get pushed onto. The promise is no longer resolved with a value, but is simply the gate gating whether or not there are any events in the queue. The iterator awaits the promise and then yields the items from the queue so long as there are any. When the queue is empty and the watch is still running, then a new promise is created and awaited upon. This whould eliminate the problem entirely and one can now go asynchronous in the loop as long as one wants without missing events.
Verified that the added test fails with v24.0.0 and passes after the fix.
Based on feedback the queuing was made configurable with the
maxQueue(default2048) option determining the maximum size of the queue and theoverflowoption deciding to eitherignorethe issue orthrowan Error (default:'ignore'). To effectively get back the previous behavior one would have to pass{ maxQueue: 1, overflow: 'ignore' }.