tls: implement tls.getCACertificates()

To accompany --use-system-ca, this adds a new API that allows
querying various kinds of CA certificates.

- If the first argument `type` is `"default"` or undefined,
  it returns the CA certificates that will be used by Node.js
  TLS clients by default, which includes the Mozilla CA
  if --use-bundled-ca is enabled or --use-openssl-ca is not
  enabled, and the system certificates if --use-system-ca
  is enabled, and the extra certificates if NODE_EXTRA_CA_CERTS
  is used.
- If `type` is `"system"` this returns the system certificates,
  regardless of whether --use-system-ca is enabeld or not.
- If `type` is `"bundled"` this is the same as `tls.rootCertificates`
  and returns the Mozilla CA certificates.
- If `type` is `"extra"` this returns the certificates parsed
  from the path specified by NODE_EXTRA_CA_CERTS.

Drive-by: remove the inaccurate description in `tls.rootCertificates`
about including system certificates, since it in fact does not include
them, and also it is contradicting the previous description about
`tls.rootCertificates` always returning the Mozilla CA store and
staying the same across platforms.

PR-URL: #57107
Reviewed-By: James M Snell <[email protected]>

Author: joyeecheung

doc/api/tls.md

@@ -1979,9 +1979,13 @@ changes:
   * `allowPartialTrustChain` {boolean} Treat intermediate (non-self-signed)
     certificates in the trust CA certificate list as trusted.
   * `ca` {string|string\[]|Buffer|Buffer\[]} Optionally override the trusted CA
-    certificates. Default is to trust the well-known CAs curated by Mozilla.
-    Mozilla's CAs are completely replaced when CAs are explicitly specified
-    using this option. The value can be a string or `Buffer`, or an `Array` of
+    certificates. If not specified, the CA certificates trusted by default are
+    the same as the ones returned by [`tls.getCACertificates()`][] using the
+    `default` type.  If specified, the default list would be completely replaced
+    (instead of being concatenated) by the certificates in the `ca` option.
+    Users need to concatenate manually if they wish to add additional certificates
+    instead of completely overriding the default.
+    The value can be a string or `Buffer`, or an `Array` of
     strings and/or `Buffer`s. Any string or `Buffer` can contain multiple PEM
     CAs concatenated together. The peer's certificate must be chainable to a CA
     trusted by the server for the connection to be authenticated. When using
@@ -1995,7 +1999,6 @@ changes:
     provided.
     For PEM encoded certificates, supported types are "TRUSTED CERTIFICATE",
     "X509 CERTIFICATE", and "CERTIFICATE".
-    See also [`tls.rootCertificates`][].
   * `cert` {string|string\[]|Buffer|Buffer\[]} Cert chains in PEM format. One
     cert chain should be provided per private key. Each cert chain should
     consist of the PEM formatted certificate for a provided private `key`,
@@ -2356,6 +2359,39 @@ openssl pkcs12 -certpbe AES-256-CBC -export -out client-cert.pem \
 The server can be tested by connecting to it using the example client from
 [`tls.connect()`][].
 
+## `tls.getCACertificates([type])`
+
+<!-- YAML
+added: REPLACEME
+-->
+
+* `type` {string|undefined} The type of CA certificates that will be returned. Valid values
+  are `"default"`, `"system"`, `"bundled"` and `"extra"`.
+  **Default:** `"default"`.
+* Returns: {string\[]} An array of PEM-encoded certificates. The array may contain duplicates
+  if the same certificate is repeatedly stored in multiple sources.
+
+Returns an array containing the CA certificates from various sources, depending on `type`:
+
+* `"default"`: return the CA certificates that will be used by the Node.js TLS clients by default.
+  * When [`--use-bundled-ca`][] is enabled (default), or [`--use-openssl-ca`][] is not enabled,
+    this would include CA certificates from the bundled Mozilla CA store.
+  * When [`--use-system-ca`][] is enabled, this would also include certificates from the system's
+    trusted store.
+  * When [`NODE_EXTRA_CA_CERTS`][] is used, this would also include certificates loaded from the specified
+    file.
+* `"system"`: return the CA certificates that are loaded from the system's trusted store, according
+  to rules set by [`--use-system-ca`][]. This can be used to get the certificates from the system
+  when [`--use-system-ca`][] is not enabled.
+* `"bundled"`: return the CA certificates from the bundled Mozilla CA store. This would be the same
+  as [`tls.rootCertificates`][].
+* `"extra"`: return the CA certificates loaded from [`NODE_EXTRA_CA_CERTS`][]. It's an empty array if
+  [`NODE_EXTRA_CA_CERTS`][] is not set.
+
+<!-- YAML
+added: v0.10.2
+-->
+
 ## `tls.getCiphers()`
 
 <!-- YAML
@@ -2392,8 +2428,10 @@ from the bundled Mozilla CA store as supplied by the current Node.js version.
 The bundled CA store, as supplied by Node.js, is a snapshot of Mozilla CA store
 that is fixed at release time. It is identical on all supported platforms.
 
-On macOS if `--use-system-ca` is passed then trusted certificates
-from the user and system keychains are also included.
+To get the actual CA certificates used by the current Node.js instance, which
+may include certificates loaded from the system store (if `--use-system-ca` is used)
+or loaded from a file indicated by `NODE_EXTRA_CA_CERTS`, use
+[`tls.getCACertificates()`][].
 
 ## `tls.DEFAULT_ECDH_CURVE`
 
@@ -2479,7 +2517,11 @@ added:
 [`'secureConnection'`]: #event-secureconnection
 [`'session'`]: #event-session
 [`--tls-cipher-list`]: cli.md#--tls-cipher-listlist
+[`--use-bundled-ca`]: cli.md#--use-bundled-ca---use-openssl-ca
+[`--use-openssl-ca`]: cli.md#--use-bundled-ca---use-openssl-ca
+[`--use-system-ca`]: cli.md#--use-system-ca
 [`Duplex`]: stream.md#class-streamduplex
+[`NODE_EXTRA_CA_CERTS`]: cli.md#node_extra_ca_certsfile
 [`NODE_OPTIONS`]: cli.md#node_optionsoptions
 [`SSL_export_keying_material`]: https://www.openssl.org/docs/man1.1.1/man3/SSL_export_keying_material.html
 [`SSL_get_version`]: https://www.openssl.org/docs/man1.1.1/man3/SSL_get_version.html
@@ -2508,6 +2550,7 @@ added:
 [`tls.createSecureContext()`]: #tlscreatesecurecontextoptions
 [`tls.createSecurePair()`]: #tlscreatesecurepaircontext-isserver-requestcert-rejectunauthorized-options
 [`tls.createServer()`]: #tlscreateserveroptions-secureconnectionlistener
+[`tls.getCACertificates()`]: #tlsgetcacertificatestype
 [`tls.getCiphers()`]: #tlsgetciphers
 [`tls.rootCertificates`]: #tlsrootcertificates
 [`x509.checkHost()`]: crypto.md#x509checkhostname-options

lib/tls.js

@@ -24,6 +24,8 @@
 const {
   Array,
   ArrayIsArray,
+  // eslint-disable-next-line no-restricted-syntax
+  ArrayPrototypePush,
   JSONParse,
   ObjectDefineProperty,
   ObjectFreeze,
@@ -34,6 +36,7 @@ const {
   ERR_TLS_CERT_ALTNAME_FORMAT,
   ERR_TLS_CERT_ALTNAME_INVALID,
   ERR_OUT_OF_RANGE,
+  ERR_INVALID_ARG_VALUE,
 } = require('internal/errors').codes;
 const internalUtil = require('internal/util');
 internalUtil.assertCrypto();
@@ -44,12 +47,18 @@ const {
 
 const net = require('net');
 const { getOptionValue } = require('internal/options');
-const { getRootCertificates, getSSLCiphers } = internalBinding('crypto');
+const {
+  getBundledRootCertificates,
+  getExtraCACertificates,
+  getSystemCACertificates,
+  getSSLCiphers,
+} = internalBinding('crypto');
 const { Buffer } = require('buffer');
 const { canonicalizeIP } = internalBinding('cares_wrap');
 const _tls_common = require('_tls_common');
 const _tls_wrap = require('_tls_wrap');
 const { createSecurePair } = require('internal/tls/secure-pair');
+const { validateString } = require('internal/validators');
 
 // Allow {CLIENT_RENEG_LIMIT} client-initiated session renegotiations
 // every {CLIENT_RENEG_WINDOW} seconds. An error event is emitted if more
@@ -85,23 +94,84 @@ exports.getCiphers = internalUtil.cachedResult(
   () => internalUtil.filterDuplicateStrings(getSSLCiphers(), true),
 );
 
-let rootCertificates;
+let bundledRootCertificates;
+function cacheBundledRootCertificates() {
+  bundledRootCertificates ||= ObjectFreeze(getBundledRootCertificates());
 
-function cacheRootCertificates() {
-  rootCertificates = ObjectFreeze(getRootCertificates());
+  return bundledRootCertificates;
 }
 
 ObjectDefineProperty(exports, 'rootCertificates', {
   __proto__: null,
   configurable: false,
   enumerable: true,
-  get: () => {
-    // Out-of-line caching to promote inlining the getter.
-    if (!rootCertificates) cacheRootCertificates();
-    return rootCertificates;
-  },
+  get: cacheBundledRootCertificates,
 });
 
+let extraCACertificates;
+function cacheExtraCACertificates() {
+  extraCACertificates ||= ObjectFreeze(getExtraCACertificates());
+
+  return extraCACertificates;
+}
+
+let systemCACertificates;
+function cacheSystemCACertificates() {
+  systemCACertificates ||= ObjectFreeze(getSystemCACertificates());
+
+  return systemCACertificates;
+}
+
+let defaultCACertificates;
+function cacheDefaultCACertificates() {
+  if (defaultCACertificates) { return defaultCACertificates; }
+  defaultCACertificates = [];
+
+  if (!getOptionValue('--use-openssl-ca')) {
+    const bundled = cacheBundledRootCertificates();
+    for (let i = 0; i < bundled.length; ++i) {
+      ArrayPrototypePush(defaultCACertificates, bundled[i]);
+    }
+    if (getOptionValue('--use-system-ca')) {
+      const system = cacheSystemCACertificates();
+      for (let i = 0; i < system.length; ++i) {
+
+        ArrayPrototypePush(defaultCACertificates, system[i]);
+      }
+    }
+  }
+
+  if (process.env.NODE_EXTRA_CA_CERTS) {
+    const extra = cacheExtraCACertificates();
+    for (let i = 0; i < extra.length; ++i) {
+
+      ArrayPrototypePush(defaultCACertificates, extra[i]);
+    }
+  }
+
+  ObjectFreeze(defaultCACertificates);
+  return defaultCACertificates;
+}
+
+// TODO(joyeecheung): support X509Certificate output?
+function getCACertificates(type = 'default') {
+  validateString(type, 'type');
+
+  switch (type) {
+    case 'default':
+      return cacheDefaultCACertificates();
+    case 'bundled':
+      return cacheBundledRootCertificates();
+    case 'system':
+      return cacheSystemCACertificates();
+    case 'extra':
+      return cacheExtraCACertificates();
+    default:
+      throw new ERR_INVALID_ARG_VALUE('type', type);
+  }
+}
+exports.getCACertificates = getCACertificates;
+
 // Convert protocols array into valid OpenSSL protocols list
 // ("\x06spdy/2\x08http/1.1\x08http/1.0")
 function convertProtocols(protocols) {

src/crypto/crypto_context.cc

@@ -42,11 +42,13 @@ using ncrypto::MarkPopErrorOnReturn;
 using ncrypto::SSLPointer;
 using ncrypto::StackOfX509;
 using ncrypto::X509Pointer;
+using ncrypto::X509View;
 using v8::Array;
 using v8::ArrayBufferView;
 using v8::Boolean;
 using v8::Context;
 using v8::DontDelete;
+using v8::EscapableHandleScope;
 using v8::Exception;
 using v8::External;
 using v8::FunctionCallbackInfo;
@@ -57,7 +59,9 @@ using v8::Integer;
 using v8::Isolate;
 using v8::JustVoid;
 using v8::Local;
+using v8::LocalVector;
 using v8::Maybe;
+using v8::MaybeLocal;
 using v8::Nothing;
 using v8::Object;
 using v8::PropertyAttribute;
@@ -672,9 +676,6 @@ static void LoadCertsFromDir(std::vector<X509*>* certs,
     return;
   }
 
-  uv_fs_t stats_req;
-  auto cleanup_stats =
-      OnScopeLeave([&stats_req]() { uv_fs_req_cleanup(&stats_req); });
   for (;;) {
     uv_dirent_t ent;
 
@@ -691,12 +692,14 @@ static void LoadCertsFromDir(std::vector<X509*>* certs,
       return;
     }
 
+    uv_fs_t stats_req;
     std::string file_path = std::string(cert_dir) + "/" + ent.name;
     int stats_r = uv_fs_stat(nullptr, &stats_req, file_path.c_str(), nullptr);
     if (stats_r == 0 &&
         (static_cast<uv_stat_t*>(stats_req.ptr)->st_mode & S_IFREG)) {
       LoadCertsFromFile(certs, file_path.c_str());
     }
+    uv_fs_req_cleanup(&stats_req);
   }
 }
 
@@ -775,7 +778,7 @@ static std::vector<X509*> InitializeSystemStoreCertificates() {
   return system_store_certs;
 }
 
-static std::vector<X509*>& GetSystemStoreRootCertificates() {
+static std::vector<X509*>& GetSystemStoreCACertificates() {
   // Use function-local static to guarantee thread safety.
   static std::vector<X509*> system_store_certs =
       InitializeSystemStoreCertificates();
@@ -847,7 +850,7 @@ X509_STORE* NewRootCertStore() {
       CHECK_EQ(1, X509_STORE_add_cert(store, cert));
     }
     if (per_process::cli_options->use_system_ca) {
-      for (X509* cert : GetSystemStoreRootCertificates()) {
+      for (X509* cert : GetSystemStoreCACertificates()) {
         CHECK_EQ(1, X509_STORE_add_cert(store, cert));
       }
     }
@@ -869,7 +872,7 @@ void CleanupCachedRootCertificates() {
     }
   }
   if (has_cached_system_root_certs.load()) {
-    for (X509* cert : GetSystemStoreRootCertificates()) {
+    for (X509* cert : GetSystemStoreCACertificates()) {
       X509_free(cert);
     }
   }
@@ -881,7 +884,7 @@ void CleanupCachedRootCertificates() {
   }
 }
 
-void GetRootCertificates(const FunctionCallbackInfo<Value>& args) {
+void GetBundledRootCertificates(const FunctionCallbackInfo<Value>& args) {
   Environment* env = Environment::GetCurrent(args);
   Local<Value> result[arraysize(root_certs)];
 
@@ -898,6 +901,58 @@ void GetRootCertificates(const FunctionCallbackInfo<Value>& args) {
       Array::New(env->isolate(), result, arraysize(root_certs)));
 }
 
+MaybeLocal<Array> X509sToArrayOfStrings(Environment* env,
+                                        const std::vector<X509*>& certs) {
+  ClearErrorOnReturn clear_error_on_return;
+  EscapableHandleScope scope(env->isolate());
+
+  LocalVector<Value> result(env->isolate(), certs.size());
+  for (size_t i = 0; i < certs.size(); ++i) {
+    X509View view(certs[i]);
+    auto pem_bio = view.toPEM();
+    if (!pem_bio) {
+      ThrowCryptoError(env, ERR_get_error(), "X509 to PEM conversion");
+      return MaybeLocal<Array>();
+    }
+
+    char* pem_data = nullptr;
+    auto pem_size = BIO_get_mem_data(pem_bio.get(), &pem_data);
+    if (pem_size <= 0 || !pem_data) {
+      ThrowCryptoError(env, ERR_get_error(), "Reading PEM data");
+      return MaybeLocal<Array>();
+    }
+    // PEM is base64-encoded, so it must be one-byte.
+    if (!String::NewFromOneByte(env->isolate(),
+                                reinterpret_cast<uint8_t*>(pem_data),
+                                v8::NewStringType::kNormal,
+                                pem_size)
+             .ToLocal(&result[i])) {
+      return MaybeLocal<Array>();
+    }
+  }
+  return scope.Escape(Array::New(env->isolate(), result.data(), result.size()));
+}
+
+void GetSystemCACertificates(const FunctionCallbackInfo<Value>& args) {
+  Environment* env = Environment::GetCurrent(args);
+  Local<Array> results;
+  if (X509sToArrayOfStrings(env, GetSystemStoreCACertificates())
+          .ToLocal(&results)) {
+    args.GetReturnValue().Set(results);
+  }
+}
+
+void GetExtraCACertificates(const FunctionCallbackInfo<Value>& args) {
+  Environment* env = Environment::GetCurrent(args);
+  if (extra_root_certs_file.empty()) {
+    return args.GetReturnValue().Set(Array::New(env->isolate()));
+  }
+  Local<Array> results;
+  if (X509sToArrayOfStrings(env, GetExtraCACertificates()).ToLocal(&results)) {
+    args.GetReturnValue().Set(results);
+  }
+}
+
 bool SecureContext::HasInstance(Environment* env, const Local<Value>& value) {
   return GetConstructorTemplate(env)->HasInstance(value);
 }
@@ -981,8 +1036,14 @@ void SecureContext::Initialize(Environment* env, Local<Object> target) {
                          GetConstructorTemplate(env),
                          SetConstructorFunctionFlag::NONE);
 
+  SetMethodNoSideEffect(context,
+                        target,
+                        "getBundledRootCertificates",
+                        GetBundledRootCertificates);
+  SetMethodNoSideEffect(
+      context, target, "getSystemCACertificates", GetSystemCACertificates);
   SetMethodNoSideEffect(
-      context, target, "getRootCertificates", GetRootCertificates);
+      context, target, "getExtraCACertificates", GetExtraCACertificates);
 }
 
 void SecureContext::RegisterExternalReferences(
@@ -1022,7 +1083,9 @@ void SecureContext::RegisterExternalReferences(
 
   registry->Register(CtxGetter);
 
-  registry->Register(GetRootCertificates);
+  registry->Register(GetBundledRootCertificates);
+  registry->Register(GetSystemCACertificates);
+  registry->Register(GetExtraCACertificates);
 }
 
 SecureContext* SecureContext::Create(Environment* env) {