Skip to content

[Tracking Issue] Web Platform Tests revamp/update automation #23192

Closed
@joyeecheung

Description

@joyeecheung

It's been a while since we update the Web Platform Tests in test/parallel in bulk and there have been some recent updates upstream that make the tests easier to run in our test suite, so here is a tracking issue for a revamp of existing tests (hopefully building the foundation of more test integration in the future).

I have a WIP in node-core-utils that currently pulls the tests/harness, track the commit hashes to a file, and makes sure that the license stuff are in place. There is also a WIP branch for this repo with the revamp for URL tests. I was thinking about writing a design doc and get some feedback before opening PRs in both repo, but probably a GitHub issue may work better so here it goes.

  • Tooling to pull tests from upstream automatically (right now they are copy-pasted by hand), maintain the commit hashes in a local file and in the git history, and makes sure the license stuff are in place. (In my WIP I put them all in test/fixtures/wpt so the linter will ignore them)
  • Have a test runner that uses the harness in WPT instead of our mocks in test/common/wpt.js
  • Sort out a status file for test expectations (including requirement about intl/crypto, and expected failures, .etc). I have implemented this in my fork with a separate format, but we may also consider changing our python test runner to consume a status file with finer-granularity of control and communicate with Jenkins nicely in the future.
  • Migrate the .any.js tests upstream into a separate test/wpt directory with a test runner that creates a sandbox with the WPT harness to run the tests and use the status file to check expectations.
  • Work with the upstream to migrate more tests out of HTML and put them into JS files that are easier for us to pull down into our repository, as well as split out the parts that cannot be run by us (involving certain APIs that we have not implemented) from the parts that we can run.

------------ TODO ----------

  • Fix existing test failures: mostly IDL failures. See status files
  • Port encoding tests (need to figure out ICU requirements for the CI): test: port encoding WPT into core #25321
  • Port timer tests and mark expected failures
  • Port performance tests and mark expected failures
  • Port messaging tests and mark expected failures
  • Upstream/split existing test/parallel/test-whatwg-url-custom-*.js and remove them
  • Upstream/split existing test/parallel/test-whatwg-encoding-custom-*.js and remove them
  • Support running .window.js (but we are not a window, so we need to add another scope for us in the upstream and potentially WebIDL)
  • Enable the driver to be ported out to another project so that it can be reused by our "official" modules: POC in [POC]: Porting WPT utilities in Node.js core whatwg-stream#2

Metadata

Metadata

Assignees

No one assigned

    Labels

    testIssues and PRs related to the tests.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions