chore NET-1404: remove deprecated package request from autocert-*

[harbu]

Summary

Package request has been deprecated since 2020. Removed its usage from sub-packages autocertifier-server and autocertifier-client and replaced with with native https module's request functionality.

Also made test packages/autocertifier-server/test/integration/RestServer-real-http-client.test.ts exercise the new HTTP requesting utility in autocertifier-client to gain more test coverage confidence.

I tested the certificate fetching process by hand by creating a new cloud instance and installing this branch there. My node was successfully able to get a subdomain and cert assigned to itself by the production autocertifier-server.

Future improvements

Use built-in node.js fetch instead of https. However, see the discussion of this PR as to how this needs to be approached.

Checklist before requesting a review

• Is this a breaking change? If it is, be clear in summary.
• Read through code myself one more time.
• Make sure any and all TODO comments left behind are meant to be left in.
• Has reasonable passing test coverage?
• Updated changelog if applicable.
• Updated documentation if applicable.

[linear]

NET-1404 Replace deprecated `request` with another HTTP client library in autocertifier packages

[harbu]

There are no tests in the autocertifier-client sub-package. How can I check that my changes work? In particular I removed the allowance of self-signed certs in the autocertifier-client, should the ability to allow self-signed certs be kept in? Ping @juslesan @ptesavol

[juslesan]

There are no tests in the autocertifier-client sub-package. How can I check that my changes work?

There is a test in the autocertifier-server that tests that the integration between the client and server works. It should check that the basic functionality is okay.

In particular I removed the allowance of self-signed certs in the autocertifier-client, should the ability to allow self-signed certs be kept in?

This is something that should be checked by running the client against the production server. I do believe there is a proper TLS certificate there so this should not be required.

[ptesavol]

Good simplification indeed, many lines of code less. However, one thing to discuss is whether we should move to using fetch already. Fetch will be available in the global object the same way as in browsers starting from NodeJS 21, and we could fall back to node-fetch package on older NodeJS versions. You can see see this post about fetch in node: https://blog.logrocket.com/fetch-api-node-js/

[harbu]

Good simplification indeed, many lines of code less. However, one thing to discuss is whether we should move to using fetch already. Fetch will be available in the global object the same way as in browsers starting from NodeJS 21, and we could fall back to node-fetch package on older NodeJS versions. You can see see this post about fetch in node: https://blog.logrocket.com/fetch-api-node-js/

I had a go at using built-in node.js fetch to accomplish this first, however there seems to be no way of passing a option to allow for self-signed certificates or to disable certificate validation. Apparently to achieve this you need to install a package undici and I'm not sure it is worth it in this case (see https://github.com/orgs/nodejs/discussions/44038#discussioncomment-5701073)