Add details about skipping tests in VCR, and when this is needed (#12407)

Author: SarahFrench

docs/content/develop/test/test.md

@@ -272,6 +272,59 @@ func TestSignatureAlgorithmDiffSuppress(t *testing.T) {
 }
 ```
 
+## Skip tests in VCR replaying mode
+
+Acceptance tests are run in VCR replaying mode on PRs (using pre-recorded HTTP requests and responses) to reduce the time it takes to present results to contributors. However, not all resources or tests are possible to run in replaying mode. Incompatible tests should be skipped during VCR replaying mode. They will still run in our nightly test suite.
+
+{{< tabs "skipping-tests-in-vcr-replaying" >}}
+
+   {{< tab "Skip a generated test" >}}
+   Skipping acceptance tests that are generated from example files can be achieved by adding `skip_vcr: true` in the example's YAML:
+
+   ```yaml
+   examples:
+   - name: 'bigtable_app_profile_anycluster'
+      ...
+
+      # bigtable instance does not use the shared HTTP client, this test creates an instance
+      skip_vcr: true
+   ```
+
+   If you skip a test in VCR mode, include a code comment explaining the reason for skipping (for example, a link to a GitHub issue.)
+
+   {{< /tab >}}
+   {{< tab "Skip a handwritten test" >}}
+   Skipping acceptance tests that are handwritten can be achieved by adding `acctest.SkipIfVcr(t)` at the start of the test:
+
+   ```go
+   func TestAccPubsubTopic_update(t *testing.T) {
+         acctest.SkipIfVcr(t) // See: https://github.com/hashicorp/terraform-provider-google/issues/9999
+         acctest.VcrTest(t, resource.TestCase{ ... }
+   }
+   ```
+
+   If you skip a test in VCR mode, include a code comment explaining the reason for skipping (for example, a link to a GitHub issue.)
+
+   {{< /tab >}}
+{{< /tabs >}}
+
+### Reasons that tests are skipped in VCR replaying mode
+
+| Problem                                          | How to fix/Other info  | Skip in VCR replaying? |
+| ------------------------------------------------ | ---------------------- |------------- |
+| *Incorrect or insufficient data is present in VCR recordings to replay tests*.  Tests will fail with `Requested interaction not found` errors during REPLAYING mode | Make sure that you're not introducing randomness into the test, e.g. by unnecessarily using the random provider to set a resource's name.| If you cannot avoid this issue you should skip the test, but try to ensure that it cannot be fixed first.|
+*Bigtable acceptance tests aren't working in VCR mode*. `Requested interaction not found` errors are seen during Bigtable tests run in REPLAYING mode | Currently the provider uses a separate client than the rest of the provider to interact with the Bigtable API. As HTTP traffic to the Bigtable API doesn't go via the shared client it cannot be recorded in RECORDING mode.| Skip the test in VCR for Bigtable. |
+| *Using multiple provider aliases doesn't work in VCR*. You may have two instances of the google provider in the test config but one of them doesn't seem to be using its provider arguments - for example, using the wrong default project. | See this GitHub issue: https://github.com/hashicorp/terraform-provider-google/issues/20019 . The problem is that, due to how the VCR system works, one provider instance will be configured and the other will be forced to reuse the first instance's configuration, despite them being given different provider arguments. |  Skip the test in VCR is using aliases is unavoidable. |
+| *Using multiple versions of the google/google-beta provider in a single test isn't working in VCR*. Unexpected test failures may occur during tests in REPLAYING mode where `ExternalProviders` is used to pull in past versions of the google/google-beta provider. | When ExternalProviders is used to pulling in other versions of the provider, any HTTP traffic through the external provider will not be recorded. If the HTTP traffic produces an unexpected result or returns an API error then the test will fail in REPLAYING mode. | Skip the test in VCR when testing the current provider behaviour versus previous released versions. |
+
+Some additional things to bear in mind are that VCR tests in REPLAYING mode will still interact with GCP APIs somewhat. For example:
+
+- When the provider is configured it will use credentials to obtain access tokens from GCP
+- Some acceptance tests use bootstrapping functions that ensure long-lived resources are present in a testing project before the provider is tested.
+
+These tests can still run in VCR replaying mode; however, REPLAYING mode can't be used as a way to completely avoid HTTP traffic generally or with GCP APIs.
+
+
 ## What's next?
 
 - [Run your tests]({{< ref "/develop/test/run-tests" >}})