build, doc: use new api doc tooling

[flakey5]

Switches over to using the new doc generation tooling. For more background on this, please see #52343

Currently a draft just to get feedback on the approach to this integration.

cc @nodejs/web-infra

[nodejs-github-bot]

Review requested:

• @nodejs/nodejs-website
• @nodejs/web-infra

[ovflowd]

@flakey5 I guess you also need to check our GitHub Action Workflows, and also other mentions of these files within the source.

Like within the Contributor Docs, there is a file that describes how the legacy API doc tooling works, and I believe there are other references also.

[flakey5]

lint-js-and-md is failing because of the linting errors since it exits with a non-zero status code if there's anything wrong with the docs. I think we should skip the REPLACEME checks for normal ci runs.

It also looks like synopsis.md fails the introduced_in check because it's not under the top-level header, should it be enforced that introduced_in goes under the top-level header or should we change it to just check that it exists somewhere in the file (like it was doing previously)?

[flakey5]

Also not sure what's going on with lint-addon-docs? cc @araujogui

[ovflowd]

lint-js-and-md is failing because of the linting errors since it exits with a non-zero status code if there's anything wrong with the docs. I think we should skip the REPLACEME checks for normal ci runs.

It also looks like synopsis.md fails the introduced_in check because it's not under the top-level header, should it be enforced that introduced_in goes under the top-level header or should we change it to just check that it exists somewhere in the file (like it was doing previously)?

REPLACEME shouldn't error, imo, just give a warning. Our linter should have warn and error levels.

And yes introduced_in must be top level!

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.09%. Comparing base (a430ef5) to head (0702f56).
Report is 81 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #57343      +/-   ##
==========================================
- Coverage   90.22%   90.09%   -0.13%     
==========================================
  Files         635      634       -1     
  Lines      187313   187264      -49     
  Branches    36784    36707      -77     
==========================================
- Hits       168998   168717     -281     
- Misses      11080    11365     +285     
+ Partials     7235     7182      -53     

see 117 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[araujogui]

lint-js-and-md is failing because of the linting errors since it exits with a non-zero status code if there's anything wrong with the docs. I think we should skip the REPLACEME checks for normal ci runs.

It also looks like synopsis.md fails the introduced_in check because it's not under the top-level header, should it be enforced that introduced_in goes under the top-level header or should we change it to just check that it exists somewhere in the file (like it was doing previously)?

Actually, the linter only returns 1 if there's an error-level issue, and I don't think that's the case here.

[araujogui]

Also not sure what's going on with lint-addon-docs? cc @araujogui

I’m not sure either, but I’ll check it out.

[ovflowd]

@flakey5:

3:1-3:9   warning Use "the Node.js" instead of "Node.js'" prohibited-strings remark-lint
4:46-4:50 warning Use "Node.js" instead of "Node"         prohibited-strings remark-lint

On the README.md file you updated (tools/doc/README.md) after updating those can you run make format-md (?)

[ovflowd]

This is the result of many months of arduous work between many awesome folks, including @flakey5 @AugustinMauroy @araujogui @ovflowd @avivkeller and others.

I'm so proud of what we are achieving here and this is a huge step towards a modern tooling and a revamped API docs within Node.js

Approving, as I believe this is ready!

[ovflowd]

cc @nodejs/collaborators can we have another approval here? 🙏

[lpinca]

RSLGTM because it is hard to review and outside of my comfort zone.

[aduh95]

@avivkeller I can see that, it's in the diff; what's not in the diff is why is it like that, how to fix it, etc.

[flakey5]

Apologizes for the delay, got a bit busy

make: *** No rule to make target 'out/doc/api/index.html'. Stop..

This should be fixed now

I'd also like to get some comment on the performance, here's the command I'm using to measure it:

The old tooling intakes each source file individually, whereas api-docs-tooling intakes all of the files so to only need to bootstrap everything once. This pr does remove caching in the Makefile of the source Markdown files, so running make docclean doc-only -j && time make doc-only -j on the old tooling only generates the docs once whereas with the new tooling it's done twice.

Regardless, the performance isn't amazing and is something we're looking into

[aduh95]

I wonder if it'd be better to let Make handle the caching and the parallelism – it would also decrease the maintenance burden of maintaining slightly different commands

Suggested change

	$(call available-node, \
	$(NPX) --prefix tools/doc api-docs-tooling generate \
	-t legacy-html-all legacy-json-all api-links \
	-i doc/api/*.md \
	-i lib/*.js \
	--ignore $(skip_apidoc_files) \
	-o out/doc/api/ \
	--no-lint \
	-c file://$(PWD)/CHANGELOG.md \
	-v $(VERSION) \
	-p $(DOC_THREADS) \
	) \
	$(MAKE) $(apidocs_html) $(apidocs_json)

[flakey5]

I'm generally -1 on this since the new tooling is designed to support handling all the files at once, but I'm either or atp. Wdyt @ovflowd?

[ovflowd]

Im not sure what is being requested here, but in general we want to generate all files…. But we should still support users that want to generate just one file or a set of files, not sure if that’s what is being suggested here. Im also not sure parallelism is a good idea 🤔 not sure the old tooling even did that, I don’t think so, but I might be wrong.

[flakey5]

in general we want to generate all files…. But we should still support users that want to generate just one file or a set of files

The current setup allows for this, but the question was more for when we're generating all files, do we want to run the tooling separately for each file vs passing them all into the -i flag

[aduh95]

I find it quite frustrating that make docserve takes forever when everything is already up-to-date. It will make me less likely to contribute to the docs, or at least I probably won't bother checking the HTML version before submitting my changes.

[ovflowd]

The concern is that make is a system used to avoid rebuilding stuff that's already built, and I don't understand why we're not taking advantage of it – and yes, it's much much slower, I shared some figures in #57343 (review).

Noted. I'm totally fine if the point here is that Make helps skip unnecessary builds — but there's still a non-trivial cost in bootstrapping the tooling itself. Even just initializing everything takes some time. We also need context from other files to generate the docs — for instance, the navigation, search index, etc. need to be built using all the doc files.

And I’m not sure Make gives us enough fine-grained control at that level. Maybe what we actually want here is for the dev mode to behave more like ISR — and reduce the overhead during initialization.

One idea could be: parse all files upfront just enough to get the base AST, and then only run generators for files requested via the web server. WDYT @flakey5 @avivkeller — is that feasible? That way we’d be leveraging something like ISR to only generate what's actually needed, and we’d still watch for file changes.

Also wondering if we could cache some pieces of this — I know some build systems already do that, so reinventing the wheel might not be worth it. But I feel just having ISR-style logic might already help a lot, since the core parser runs relatively fast…

[avivkeller]

Maybe we do all the serving from tooling? That way, we can leverage whatever ISR-like capabilities we want?

But, I don't think that should block this, it's a QoL improvement, not something fundamental for distribution.

[ovflowd]

I'd argue about the blockability; We're changing to a new tooling with the promise of improved DevEx. I don't want to ship something in a state @aduh95 doesn't feel improves DevEx.

[aduh95]

there's still a non-trivial cost in bootstrapping the tooling itself

I feel like I'd be more than happy to pay this cost if that means I save 60s each time I run make docserve 😅

And I’m not sure Make gives us enough fine-grained control at that level

It does as long as you can store the intermediate info in intermediate files – but I'm not saying that necessarily the way to go.

[avivkeller]

I suggest we go with the --select method I outlined in the linked issue. This approach lets us keep the Makefile-style dependency management we use here—so we only rebuild the docs that have changed—while still supporting features that require access to all docs, like the sidebar.

It will involve some redundant parsing, since each doc will be processed individually with all the docs as context, but it should still be faster than the current 60s @aduh95 mentioned. Plus, we can add caching to store intermediate metadata and avoid unnecessary reprocessing.

But, we could also re invent the wheel a bit and add a --skip-unchanged like flag to skip unchanged files.

[flakey5]

Hmmm -c $(call available-node, -p 'url.pathToFileURL("CHANGELOG.md")') \ is causing this to be thrown: /bin/sh: 4: Syntax error: "then" unexpected (expecting "fi")

[aduh95]

Hmmm -c $(call available-node, -p 'url.pathToFileURL("CHANGELOG.md")') \ is causing this to be thrown: /bin/sh: 4: Syntax error: "then" unexpected (expecting "fi")

The fix would be to have the tooling take a path rather than a URL

[ovflowd]

cc @aduh95 are we good to merge?

[ovflowd]

(or is something still failing?)

[flakey5]

cc @aduh95 are we good to merge?

bump 😄

[aduh95]

I'm getting the following error when I try to generate several files:

Error: EEXIST: file already exists, mkdir '/Users/duhamean/Documents/node/out/doc/api/assets'
    at async mkdir (node:internal/fs/promises:857:10)
    at async mkDirAndCopy (node:internal/fs/cp/cp:308:3)
    at async Object.generate (file:///Users/duhamean/Documents/node/tools/doc/node_modules/@node-core/api-docs-tooling/src/generators/legacy-html/index.mjs:180:7)
Makefile:827: warning: pattern recipe did not update peer target 'out/doc/api/http2.json'.
make[1]: *** [Makefile:827: out/doc/api/embedding.html] Error 1

[ovflowd]

I'm getting the following error when I try to generate several files:

Error: EEXIST: file already exists, mkdir '/Users/duhamean/Documents/node/out/doc/api/assets'
    at async mkdir (node:internal/fs/promises:857:10)
    at async mkDirAndCopy (node:internal/fs/cp/cp:308:3)
    at async Object.generate (file:///Users/duhamean/Documents/node/tools/doc/node_modules/@node-core/api-docs-tooling/src/generators/legacy-html/index.mjs:180:7)
Makefile:827: warning: pattern recipe did not update peer target 'out/doc/api/http2.json'.
make[1]: *** [Makefile:827: out/doc/api/embedding.html] Error 1

@flakey5 any idea here? Is this trying to generate all the files at the same time? Individual file generation should still work, no?