test: improve the WPT runner and rename test files by joyeecheung · Pull Request #24826 · nodejs/node

joyeecheung · 2018-12-04T10:52:13Z

In preparation of porting other parts of WPT into test/wpt (I am currently working on performance-timeline and encoding).

test: remove reference to whatwg in file names under test/wpt

WPT covers standards in both W3C and WHATWG, as such it would be
strange to make this disparity explicit in our file names
(e.g. when testing standards that are solely in W3C, like
performance-timeline). Remove the reference to WHATWG will
also make the file names shorter.

Refs: #24823

test: improve WPT runner name matching

This patch:

Support wildcards(*) in WPT runner name matching (needed by e.g.
encoding where all the tests requires i18n support in the build)
Print failure reasons when encountering an expected failure
Fix a bug in copyGlobalsFromObject (previously it copies
properties from global instead of the given obj)

Previously an expected failure is printed as

[EXPECTED_FAILURE] response.formData() with input: %61+%4d%4D=

Now it is printed as

[EXPECTED_FAILURE] response.formData() with input: %61+%4d%4D=
missing Request and Response

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

WPT covers standards in both W3C and WHATWG, as such it would be strange to make this disparity explicit in our file names (e.g. when testing standards that are solely in W3C, like performance-timeline). Remove the reference to WHATWG will also make the file names shorter.

This patch: - Support wildcards(*) in WPT runner name matching (needed by e.g. encoding where all the tests requires i18n support in the build) - Print failure reasons when encountering an expected failure - Fix a bug in copyGlobalsFromObject (previously it copies properties from `global` instead of the given `obj`) Previously an expected failure is printed as ``` [EXPECTED_FAILURE] response.formData() with input: %61+%4d%4D= ``` Now it is printed as ``` [EXPECTED_FAILURE] response.formData() with input: %61+%4d%4D= missing Request and Response ```

nodejs-github-bot · 2018-12-04T10:52:15Z

@joyeecheung build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/1823/pipeline

joyeecheung · 2018-12-04T10:52:52Z

cc @nodejs/testing @nodejs/url

joyeecheung · 2018-12-05T10:11:38Z

CI: https://ci.nodejs.org/job/node-test-pull-request/19220/

Trott · 2018-12-06T03:08:40Z

CI: https://ci.nodejs.org/job/node-test-pull-request/19250/

Trott · 2018-12-06T05:12:08Z

Resume Build CI: https://ci.nodejs.org/job/node-test-pull-request/19262/

Trott · 2018-12-06T07:39:02Z

Failure looks related?

https://ci.nodejs.org/job/node-test-commit-linux-containered/9206/nodes=ubuntu1604_sharedlibs_withoutintl_x64/testReport/

/home/iojs/build/workspace/node-test-commit-linux-containered/test/common/wpt.js:451
      reason: reasons.join('; ')
                      ^

TypeError: reasons.join is not a function
    at WPTRunner.skip (/home/iojs/build/workspace/node-test-commit-linux-containered/test/common/wpt.js:451:23)
    at WPTRunner.buildQueue (/home/iojs/build/workspace/node-test-commit-linux-containered/test/common/wpt.js:503:14)
    at WPTRunner.runJsTests (/home/iojs/build/workspace/node-test-commit-linux-containered/test/common/wpt.js:265:20)
    at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/wpt/test-url.js:19:8)
    at Module._compile (internal/modules/cjs/loader.js:723:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:734:10)
    at Module.load (internal/modules/cjs/loader.js:620:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:560:12)
    at Function.Module._load (internal/modules/cjs/loader.js:552:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:776:12)

mcollina

LGTM

joyeecheung · 2018-12-14T18:26:22Z

(Probably there are some loopholes in the logic for a nointl build, I'll investigate)

joyeecheung · 2018-12-14T20:23:09Z

This should fix the nointl failure. CI: https://ci.nodejs.org/job/node-test-pull-request/19539/ ✔️

Trott · 2018-12-15T02:05:09Z

Landed in 87006be...b520254

WPT covers standards in both W3C and WHATWG, as such it would be strange to make this disparity explicit in our file names (e.g. when testing standards that are solely in W3C, like performance-timeline). Remove the reference to WHATWG will also make the file names shorter. PR-URL: nodejs#24826 Refs: nodejs#24823 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

This patch: - Support wildcards(*) in WPT runner name matching (needed by e.g. encoding where all the tests requires i18n support in the build) - Print failure reasons when encountering an expected failure - Fix a bug in copyGlobalsFromObject (previously it copies properties from `global` instead of the given `obj`) Previously an expected failure is printed as ``` [EXPECTED_FAILURE] response.formData() with input: %61+%4d%4D= ``` Now it is printed as ``` [EXPECTED_FAILURE] response.formData() with input: %61+%4d%4D= missing Request and Response ``` PR-URL: nodejs#24826 Refs: nodejs#24823 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

WPT covers standards in both W3C and WHATWG, as such it would be strange to make this disparity explicit in our file names (e.g. when testing standards that are solely in W3C, like performance-timeline). Remove the reference to WHATWG will also make the file names shorter. PR-URL: #24826 Refs: #24823 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

This patch: - Support wildcards(*) in WPT runner name matching (needed by e.g. encoding where all the tests requires i18n support in the build) - Print failure reasons when encountering an expected failure - Fix a bug in copyGlobalsFromObject (previously it copies properties from `global` instead of the given `obj`) Previously an expected failure is printed as ``` [EXPECTED_FAILURE] response.formData() with input: %61+%4d%4D= ``` Now it is printed as ``` [EXPECTED_FAILURE] response.formData() with input: %61+%4d%4D= missing Request and Response ``` PR-URL: #24826 Refs: #24823 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

WPT covers standards in both W3C and WHATWG, as such it would be strange to make this disparity explicit in our file names (e.g. when testing standards that are solely in W3C, like performance-timeline). Remove the reference to WHATWG will also make the file names shorter. PR-URL: nodejs#24826 Refs: nodejs#24823 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

This patch: - Support wildcards(*) in WPT runner name matching (needed by e.g. encoding where all the tests requires i18n support in the build) - Print failure reasons when encountering an expected failure - Fix a bug in copyGlobalsFromObject (previously it copies properties from `global` instead of the given `obj`) Previously an expected failure is printed as ``` [EXPECTED_FAILURE] response.formData() with input: %61+%4d%4D= ``` Now it is printed as ``` [EXPECTED_FAILURE] response.formData() with input: %61+%4d%4D= missing Request and Response ``` PR-URL: nodejs#24826 Refs: nodejs#24823 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

WPT covers standards in both W3C and WHATWG, as such it would be strange to make this disparity explicit in our file names (e.g. when testing standards that are solely in W3C, like performance-timeline). Remove the reference to WHATWG will also make the file names shorter. PR-URL: nodejs#24826 Refs: nodejs#24823 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

This patch: - Support wildcards(*) in WPT runner name matching (needed by e.g. encoding where all the tests requires i18n support in the build) - Print failure reasons when encountering an expected failure - Fix a bug in copyGlobalsFromObject (previously it copies properties from `global` instead of the given `obj`) Previously an expected failure is printed as ``` [EXPECTED_FAILURE] response.formData() with input: %61+%4d%4D= ``` Now it is printed as ``` [EXPECTED_FAILURE] response.formData() with input: %61+%4d%4D= missing Request and Response ``` PR-URL: nodejs#24826 Refs: nodejs#24823 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

joyeecheung added 2 commits December 4, 2018 17:44

nodejs-github-bot added the test Issues and PRs related to the tests. label Dec 4, 2018

mcollina approved these changes Dec 14, 2018

View reviewed changes

joyeecheung added the blocked PRs that are blocked by other issues or PRs. label Dec 14, 2018

fixup! test: improve WPT runner name matching

8671556

watilde approved these changes Dec 14, 2018

View reviewed changes

joyeecheung added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. and removed blocked PRs that are blocked by other issues or PRs. labels Dec 14, 2018

Trott closed this Dec 15, 2018

BethGriggs mentioned this pull request Dec 18, 2018

V11.5.0 proposal #25102

Merged

beevelop mentioned this pull request Dec 18, 2018

2018-12-18 Version 11.5.0 (Current) @BethGriggs beevelop/docker-android-nodejs#272

Closed

BethGriggs added the backport-requested-v10.x label Jul 3, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test: improve the WPT runner and rename test files#24826

test: improve the WPT runner and rename test files#24826
joyeecheung wants to merge 3 commits intonodejs:masterfrom
joyeecheung:wpt-improv

joyeecheung commented Dec 4, 2018 •

edited

Loading

Uh oh!

nodejs-github-bot commented Dec 4, 2018

Uh oh!

joyeecheung commented Dec 4, 2018

Uh oh!

joyeecheung commented Dec 5, 2018

Uh oh!

Trott commented Dec 6, 2018

Uh oh!

Trott commented Dec 6, 2018

Uh oh!

Trott commented Dec 6, 2018

Uh oh!

mcollina left a comment

Uh oh!

joyeecheung commented Dec 14, 2018

Uh oh!

joyeecheung commented Dec 14, 2018 •

edited by Trott

Loading

Uh oh!

Trott commented Dec 15, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

joyeecheung commented Dec 4, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

test: remove reference to whatwg in file names under test/wpt

test: improve WPT runner name matching

Checklist

Uh oh!

nodejs-github-bot commented Dec 4, 2018

Uh oh!

joyeecheung commented Dec 4, 2018

Uh oh!

joyeecheung commented Dec 5, 2018

Uh oh!

Trott commented Dec 6, 2018

Uh oh!

Trott commented Dec 6, 2018

Uh oh!

Trott commented Dec 6, 2018

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

joyeecheung commented Dec 14, 2018

Uh oh!

joyeecheung commented Dec 14, 2018 • edited by Trott Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Trott commented Dec 15, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

joyeecheung commented Dec 4, 2018 •

edited

Loading

joyeecheung commented Dec 14, 2018 •

edited by Trott

Loading