Skip to content

test: mark empty udp tests flaky on OS X#31936

Closed
sam-github wants to merge 1 commit intonodejs:masterfrom
sam-github:catalina-udp-flakes
Closed

test: mark empty udp tests flaky on OS X#31936
sam-github wants to merge 1 commit intonodejs:masterfrom
sam-github:catalina-udp-flakes

Conversation

@sam-github
Copy link
Contributor

@sam-github sam-github commented Feb 24, 2020 •
edited
Loading

They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: #30030
Refs: nodejs/build#2189 (comment)

Note: CI doesn't test on Catalina, yet (though I assume nodejs developers do). Test failures will indirectly block fixing of #29216, which requires running notarization on Catalina, which requires tests passing on Catalina.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@sam-github
test: mark empty udp tests flaky on OS X
e8667bb 
They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: nodejs#30030
Refs: nodejs/build#2189 (comment)
@nodejs-github-bot nodejs-github-bot added the test Issues and PRs related to the tests. label Feb 24, 2020
@sam-github sam-github mentioned this pull request Feb 24, 2020
Merged
@sam-github sam-github added the flaky-test Issues and PRs related to the tests with unstable failures on the CI. label Feb 24, 2020
@sam-github
Copy link
Contributor Author

sam-github commented Feb 24, 2020

I don't have access to a machine I can repro this on, but I asked @AshCripps to try it (nodejs/build#2189 (comment)). @lpinca , perhaps you can review, too?

@sam-github sam-github requested a review from lpinca February 24, 2020 18:39
@lpinca
Copy link
Member

lpinca commented Feb 24, 2020 •
edited
Loading

I've run make -j8 test with the patch applied but the process still exits with 2 not sure if expected. All the tested listed here timeout as expected.

AshCripps
AshCripps approved these changes Feb 25, 2020
View reviewed changes
Copy link
Member

@AshCripps AshCripps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me. @lpinca you have to run the tests manually using the python script to skip flaky-tests like the ci does:

bash-3.2$ tools/test.py --flaky-tests=skip
[07:30|% 100|+ 2790|-   0]: Done
bash-3.2$

@lpinca
Copy link
Member

lpinca commented Feb 25, 2020

Oh, I forgot to use the --flaky-tests flag.

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Feb 25, 2020

CI: https://ci.nodejs.org/job/node-test-pull-request/29350/

@sam-github
Copy link
Contributor Author

sam-github commented Feb 25, 2020

@AshCripps would it be helpful to get this fast-tracked? I kicked of a full CI on it, but it still needs 25 hours, but we can fast-track given its triviality and impact on CI stability if its blocking bringing the OS X 10.15/Catalina machines into CI.

@AshCripps
Copy link
Member

AshCripps commented Feb 25, 2020

@sam-github Fast track would be useful, pretty sure every release line will fail these tests when 10.15 is added to CI

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Feb 25, 2020

CI: https://ci.nodejs.org/job/node-test-pull-request/29351/

@sam-github
Copy link
Contributor Author

sam-github commented Feb 25, 2020

@nodejs/build @nodejs/testing Can we fast-track marking these tests flaky?

@sam-github sam-github added the fast-track PRs that do not need to wait for 48 hours to land. label Feb 25, 2020
@sam-github
Copy link
Contributor Author

sam-github commented Feb 25, 2020

Btw, the build status reported is incorrect - PR passed on everything but arm, as would be expected. I kicked a resume on the PR, though.

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Feb 26, 2020

CI: https://ci.nodejs.org/job/node-test-pull-request/29367/

@sam-github
Copy link
Contributor Author

sam-github commented Feb 26, 2020

Landed in 24a4f76

@sam-github sam-github closed this Feb 26, 2020
sam-github added a commit that referenced this pull request Feb 26, 2020
@sam-github
test: mark empty udp tests flaky on OS X
24a4f76 
They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: #30030
Refs: nodejs/build#2189 (comment)

PR-URL: #31936
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Beth Griggs <Bethany.Griggs@uk.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
BethGriggs pushed a commit that referenced this pull request Feb 27, 2020
@sam-github @BethGriggs
test: mark empty udp tests flaky on OS X
9bd1317 
They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: #30030
Refs: nodejs/build#2189 (comment)

PR-URL: #31936
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Beth Griggs <Bethany.Griggs@uk.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
codebytere pushed a commit that referenced this pull request Feb 27, 2020
@sam-github @codebytere
test: mark empty udp tests flaky on OS X
f89fb27 
They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: #30030
Refs: nodejs/build#2189 (comment)

PR-URL: #31936
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Beth Griggs <Bethany.Griggs@uk.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
@codebytere codebytere mentioned this pull request Feb 29, 2020
Merged
@sam-github sam-github deleted the catalina-udp-flakes branch March 6, 2020 19:04
@richardlau richardlau mentioned this pull request Mar 7, 2020
Closed
4 tasks
puzpuzpuz pushed a commit to puzpuzpuz/node that referenced this pull request Mar 8, 2020
@sam-github @puzpuzpuz
test: mark empty udp tests flaky on OS X
0b5fda0 
They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: nodejs#30030
Refs: nodejs/build#2189 (comment)

PR-URL: nodejs#31936
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Beth Griggs <Bethany.Griggs@uk.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
@puzpuzpuz puzpuzpuz mentioned this pull request Mar 8, 2020
Closed
3 tasks
@BethGriggs BethGriggs mentioned this pull request Mar 12, 2020
Merged
codebytere pushed a commit that referenced this pull request Mar 14, 2020
@sam-github @codebytere
test: mark empty udp tests flaky on OS X
5a537ba 
They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: #30030
Refs: nodejs/build#2189 (comment)

PR-URL: #31936
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Beth Griggs <Bethany.Griggs@uk.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>

PR-URL: #32146
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Vladimir de Turckheim <vlad2t@hotmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Shelley Vohr <codebytere@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jasnell jasnell jasnell approved these changes

@lpinca lpinca lpinca approved these changes

@BethGriggs BethGriggs BethGriggs approved these changes

+2 more reviewers

@rvagg rvagg rvagg approved these changes

@AshCripps AshCripps AshCripps approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

fast-track PRs that do not need to wait for 48 hours to land. flaky-test Issues and PRs related to the tests with unstable failures on the CI. test Issues and PRs related to the tests.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@sam-github @lpinca @nodejs-github-bot @AshCripps @jasnell @rvagg @BethGriggs @targos