gateway: Allow use of APQ when querying downstream services. (apollographql/apollo-server#3744)

* gateway: Allow use of APQ when querying downstream services.

This introduces support to Apollo Gateway which can leverage [Automated
Persisted Queries] (APQ) when communicating with downstream
implementing services in a federated architecture.

In an identical way as APQ saves bytes on the wire (read: network) in a
traditional handshake between a client and the server, this implements that behavior
in the internal communication between federated servers and the gateway that
fronts them.

This is accomplished by first attempting to utilizing a SHA-256 hex hash
(consistently, 32 bytes) of the operation which is destined for the
downstream server, rather than the complete, typically much larger query
itself.

In the event that the downstream server supports APQ (Apollo Server does by
default, unless it's been disabled), the downstream server will check its
APQ registry for the full operation body.  If it has it, it will use that
cached body to process the request and return the results.  If it does not
find the query in its existing registry, it will return a message to the
gateway that the query is not found in its registry, and the gateway will
re-transmit the request with the full operation payload.  On receipt of this
full query, the downstream server will cache the operation for future
requests (keyed by the SHA-256 hash), and return the expected result without
further delay.

This means that once a server has warmed up its APQ registry with repeated
operations, subsequent network chatter will be greatly reduced.
Furthermore, as noted in the attached documentation, the APQ registry can be
backed by a distributed store, such as Memcached or Redis, allowing multiple
downstream GraphQL servers to share the same cache, and persist it across
restarts.

By default, APQ behavior is disabled on `RemoteGraphQLDataSource`.  To
enable it, the `apq` property should be set to true.  Future versions of the
`RemoteGraphQLDataSource` could negotiate this capability on their own, but
this does not attempt to make that negotiation right now.

[Automated Persisted Queries]: https://www.apollographql.com/docs/apollo-server/performance/apq/

* tests: De-compose `toHaveFetched` in preparation for similar matchers.

* tests: Introduce `toHaveFetchNth` with to test critical order of fetches.

It's plausible that we should change every existing `toHaveFetch` to use
this matcher which enforces order.   Though it also seems plausible that
custom matchers aren't as flexible as they need to be in practice, since in
addition to the need to use Jest-built in methods (like the `nth`-call
matchers) there are other specific usages of this which are just surfacing
now (with APQ) that could be tested less precisely using
`expect.objectContaining()`, rather than testing concerns which are not
really necessary (like matching the hash).

If this matcher still supported partial matches then this would be possible.
However, since we're serializing the body into an `Request` before matching it
(which I'm not sure why we do and there is no comment to indicate why) this
isn't possible as Jest's matchers cannot survive that serialization.

* tests: Make `Matcher` declaration merges match new Jest definitions.

Without this change, every single usage of our `Matcher` is represented as a
type error since Jest has changed their own implementation of `Matcher` to
introduce a new generic type argument.

It's too bad that this didn't fail tests at the time that that Jest package
was updated!

* Re-jigger `RemoteGraphQLDataSource`'s `process` with new `sendRequest` method.

This introduces a new private `sendRequest` method that handles existing
behavior which existed within `RemoteGraphQLDataSource`'s `process`.

It should be a no-op change.

An upcoming commit will make multiple requests to downstream services in its
course of attempting APQ negotiation and this should facilitate that change
and avoid repetition of logic.

Apollo-Orig-Commit-AS: apollographql/apollo-server@7a8826a

Author: abernix

gateway-js/CHANGELOG.md

@@ -5,6 +5,7 @@
 > The changes noted within this `vNEXT` section have not been released yet.  New PRs and commits which introduce changes should include an entry in this `vNEXT` section as part of their development.  When a release is being prepared, a new header will be (manually) created below and the the appropriate changes within that release will be moved into the new section.
 
 - __BREAKING__: The behavior and signature of `RemoteGraphQLDataSource`'s `didReceiveResponse` method has been changed.  No changes are necessary _unless_ your implementation has overridden the default behavior of this method by either extending the class and overriding the method or by providing `didReceiveResponse` as a parameter to the `RemoteGraphQLDataSource`'s constructor options.  Implementations which have provided their own `didReceiveResponse` using either of these methods should view the PR linked here for details on what has changed.  [PR #3743](https://github.com/apollographql/apollo-server/pull/3743)
+- __NEW__: Setting the `apq` option to `true` on the `RemoteGraphQLDataSource` will enable the use of [automated persisted queries (APQ)](https://www.apollographql.com/docs/apollo-server/performance/apq/) when sending queries to downstream services.  Depending on the complexity of queries sent to downstream services, this technique can greatly reduce the size of the payloads being transmitted over the network.  Downstream implementing services must also support APQ functionality to participate in this feature (Apollo Server does by default unless it has been explicitly disabled).  As with normal APQ behavior, a downstream server must have received and registered a query once before it will be able to serve an APQ request. [#3744](https://github.com/apollographql/apollo-server/pull/3744)
 
 ## v0.12.0
 

gateway-js/src/__tests__/matchers/toCallService.ts

@@ -6,7 +6,7 @@ const prettyFormat = require('pretty-format');
 
 declare global {
   namespace jest {
-    interface Matchers<R> {
+    interface Matchers<R, T> {
       toCallService(service: string): R;
     }
   }

gateway-js/src/__tests__/matchers/toHaveBeenCalledBefore.ts

@@ -3,7 +3,7 @@
 export {};
 declare global {
   namespace jest {
-    interface Matchers<R> {
+    interface Matchers<R, T> {
       toHaveBeenCalledBefore(spy: SpyInstance): R;
     }
   }

gateway-js/src/__tests__/matchers/toHaveFetched.ts

@@ -5,18 +5,16 @@ import { Request, RequestInit, Headers } from 'apollo-server-env';
 export {};
 declare global {
   namespace jest {
-    interface Matchers<R> {
+    interface Matchers<R, T> {
       toHaveFetched(spy: SpyInstance): R;
     }
   }
 }
 
-function toHaveFetched(
-  this: jest.MatcherUtils,
-  fetch: jest.SpyInstance,
-  request: RequestInit & { url: string },
-): { message(): string; pass: boolean } {
-  let headers = new Headers();
+type ExtendedRequest = RequestInit & { url: string };
+
+function prepareHttpRequest(request: ExtendedRequest): Request {
+  const headers = new Headers();
   headers.set('Content-Type', 'application/json');
   if (request.headers) {
     for (let name in request.headers) {
@@ -30,8 +28,15 @@ function toHaveFetched(
     body: JSON.stringify(request.body),
   };
 
-  const httpRequest = new Request(request.url, options);
+  return new Request(request.url, options);
+}
 
+function toHaveFetched(
+  this: jest.MatcherUtils,
+  fetch: jest.SpyInstance,
+  request: ExtendedRequest,
+): { message(): string; pass: boolean } {
+  const httpRequest = prepareHttpRequest(request);
   let pass = false;
   let message = () => '';
   try {
@@ -47,6 +52,30 @@ function toHaveFetched(
   };
 }
 
+function toHaveFetchedNth(
+  this: jest.MatcherUtils,
+  fetch: jest.SpyInstance,
+  nthCall: number,
+  request: ExtendedRequest,
+): { message(): string; pass: boolean } {
+  const httpRequest = prepareHttpRequest(request);
+  let pass = false;
+  let message = () => '';
+  try {
+    expect(fetch).toHaveBeenNthCalledWith(nthCall, httpRequest);
+    pass = true;
+  } catch (e) {
+    message = () => e.message;
+  }
+
+  return {
+    message,
+    pass,
+  };
+}
+
+
 expect.extend({
   toHaveFetched,
+  toHaveFetchedNth,
 });

gateway-js/src/__tests__/matchers/toMatchAST.ts

@@ -3,7 +3,7 @@ const diff = require('jest-diff');
 
 declare global {
   namespace jest {
-    interface Matchers<R> {
+    interface Matchers<R, T> {
       toMatchAST(expected: ASTNode): R;
     }
   }

gateway-js/src/datasources/RemoteGraphQLDataSource.ts

@@ -12,12 +12,12 @@ import {
 import {
   fetch,
   Request,
-  RequestInit,
   Headers,
   Response,
 } from 'apollo-server-env';
 import { isObject } from '../utilities/predicates';
 import { GraphQLDataSource } from './types';
+import createSHA from 'apollo-server-core/dist/utils/createSHA';
 
 export class RemoteGraphQLDataSource implements GraphQLDataSource {
   constructor(
@@ -32,6 +32,26 @@ export class RemoteGraphQLDataSource implements GraphQLDataSource {
 
   url!: string;
 
+  /**
+   * Whether the downstream request should be made with automated persisted
+   * query (APQ) behavior enabled.
+   *
+   * @remarks When enabled, the request to the downstream service will first be
+   * attempted using a SHA-256 hash of the operation rather than including the
+   * operation itself. If the downstream server supports APQ and has this
+   * operation registered in its APQ storage, it will be able to complete the
+   * request without the entirety of the operation document being transmitted.
+   *
+   * In the event that the downstream service is unaware of the operation, it
+   * will respond with an `PersistedQueryNotFound` error and it will be resent
+   * with the full operation body for fulfillment.
+   *
+   * Generally speaking, when the downstream server is processing similar
+   * operations repeatedly, APQ can offer substantial network savings in terms
+   * of bytes transmitted over the wire between gateways and downstream servers.
+   */
+  apq: boolean = false;
+
   async process<TContext>({
     request,
     context,
@@ -52,19 +72,77 @@ export class RemoteGraphQLDataSource implements GraphQLDataSource {
       await this.willSendRequest({ request, context });
     }
 
-    const { http, ...graphqlRequest } = request;
-    const options: RequestInit = {
-      ...http,
-      body: JSON.stringify(graphqlRequest),
-    };
+    if (!request.query) {
+      throw new Error("Missing query");
+    }
+
+    const apqHash = createSHA('sha256')
+       .update(request.query)
+       .digest('hex');
 
-    const httpRequest = new Request(request.http.url, options);
+    const { query, ...requestWithoutQuery } = request;
 
     const respond = (response: GraphQLResponse, request: GraphQLRequest) =>
       typeof this.didReceiveResponse === "function"
         ? this.didReceiveResponse({ response, request, context })
         : response;
 
+    if (this.apq) {
+      // Take the original extensions and extend them with
+      // the necessary "extensions" for APQ handshaking.
+      requestWithoutQuery.extensions = {
+        ...request.extensions,
+        persistedQuery: {
+          version: 1,
+          sha256Hash: apqHash,
+        },
+      };
+
+      const apqOptimisticResponse =
+        await this.sendRequest(requestWithoutQuery, context);
+
+      // If we didn't receive notice to retry with APQ, then let's
+      // assume this is the best result we'll get and return it!
+      if (
+        !apqOptimisticResponse.errors ||
+        !apqOptimisticResponse.errors.find(error =>
+          error.message === 'PersistedQueryNotFound')
+      ) {
+        return respond(apqOptimisticResponse, requestWithoutQuery);
+      }
+    }
+
+    // If APQ was enabled, we'll run the same request again, but add in the
+    // previously omitted `query`.  If APQ was NOT enabled, this is the first
+    // request (non-APQ, all the way).
+    const requestWithQuery: GraphQLRequest = {
+      query,
+      ...requestWithoutQuery,
+    };
+    const response = await this.sendRequest(requestWithQuery, context);
+    return respond(response, requestWithQuery);
+  }
+
+  private async sendRequest<TContext>(
+    request: GraphQLRequest,
+    context: TContext,
+  ): Promise<GraphQLResponse> {
+
+    // This would represent an internal programming error since this shouldn't
+    // be possible in the way that this method is invoked right now.
+    if (!request.http) {
+      throw new Error("Internal error: Only 'http' requests are supported.")
+    }
+
+    // We don't want to serialize the `http` properties into the body that is
+    // being transmitted.  Instead, we want those to be used to indicate what
+    // we're accessing (e.g. url) and what we access it with (e.g. headers).
+    const { http, ...requestWithoutHttp } = request;
+    const httpRequest = new Request(http.url, {
+      ...http,
+      body: JSON.stringify(requestWithoutHttp),
+    });
+
     try {
       const httpResponse = await fetch(httpRequest);
 
@@ -78,12 +156,10 @@ export class RemoteGraphQLDataSource implements GraphQLDataSource {
         throw new Error(`Expected JSON response body, but received: ${body}`);
       }
 
-      const response: GraphQLResponse = {
+      return {
         ...body,
         http: httpResponse,
       };
-
-      return respond(response, request);
     } catch (error) {
       this.didEncounterError(error, httpRequest);
       throw error;