add `google_kms_crypto_key_version_latest` #11381

BBBmau · 2024-08-06T22:37:56Z

Currently their is no way for users to grab the latest crypto key version without needing to use a separate resource that can perform a get of cryptoKeyVersions in order to grab the latest version from a latest.

This PR addes a crypto_key_version data source that takes in a crypto_key id in order to output the latest cryptoVersion for the specified crypto_key

simple tfconfig:

resource "google_kms_key_ring" "key_ring" {
  project  = "<INSERT-PROJECT>"
  name     = "<INSERT-KEY-RING>"
  location = "us-central1"
}

resource "google_kms_crypto_key" "crypto_key" {
  name     = "<INSERT-CRYPTO-KEY>"
  key_ring = google_kms_key_ring.key_ring.id
  purpose  = "ASYMMETRIC_SIGN"

  version_template {
    algorithm = "EC_SIGN_P256_SHA256"
  }
}

data "google_kms_crypto_key_version_latest" "latest_version_data_source" {
  crypto_key = google_kms_crypto_key.crypto_key.id
}

output "latest_version" {
  value = data.google_kms_crypto_key_version_latest.latest_version_data_source.version
}

The GET request of cryptoVersions:

 ---[ REQUEST ]---------------------------------------
 GET /v1/projects/hc-terraform-testing/locations/us-central1/keyRings/mau-latest-test/cryptoKeys/mau-crypto-key-test/cryptoKeyVersions?alt=json HTTP/1.1
 Host: cloudkms.googleapis.com
 User-Agent: Terraform/1.8.0-alpha20240131 (+https://www.terraform.io) Terraform-Plugin-SDK/2.33.0 terraform-provider-google/dev
 Content-Type: application/json
 Accept-Encoding: gzip


 -----------------------------------------------------
 2024/08/06 15:27:21 [DEBUG] Google API Response Details:
 ---[ RESPONSE ]--------------------------------------
 HTTP/2.0 200 OK
 Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
 Cache-Control: private
 Content-Type: application/json; charset=UTF-8
 Date: Tue, 06 Aug 2024 22:27:21 GMT
 Server: ESF
 Vary: Origin
 Vary: X-Origin
 Vary: Referer
 X-Content-Type-Options: nosniff
 X-Frame-Options: SAMEORIGIN
 X-Xss-Protection: 0

 {
   "cryptoKeyVersions": [
     {
       "name": "projects/hc-terraform-testing/locations/us-central1/keyRings/mau-latest-test/cryptoKeys/mau-crypto-key-test/cryptoKeyVersions/1",
       "state": "DESTROY_SCHEDULED",
       "createTime": "2024-08-01T22:50:13.684360401Z",
       "destroyTime": "2024-08-31T23:07:49.100169Z",
       "protectionLevel": "SOFTWARE",
       "algorithm": "EC_SIGN_P256_SHA256",
       "generateTime": "2024-08-01T22:50:13.757083972Z"
     },
     {
       "name": "projects/hc-terraform-testing/locations/us-central1/keyRings/mau-latest-test/cryptoKeys/mau-crypto-key-test/cryptoKeyVersions/2",
       "state": "ENABLED",
       "createTime": "2024-08-06T20:20:32.503393705Z",
       "protectionLevel": "SOFTWARE",
       "algorithm": "EC_SIGN_P256_SHA256",
       "generateTime": "2024-08-06T20:20:32.539700895Z"
     },
     {
       "name": "projects/hc-terraform-testing/locations/us-central1/keyRings/mau-latest-test/cryptoKeys/mau-crypto-key-test/cryptoKeyVersions/3",
       "state": "ENABLED",
       "createTime": "2024-08-06T20:20:35.716831348Z",
       "protectionLevel": "SOFTWARE",
       "algorithm": "EC_SIGN_P256_SHA256",
       "generateTime": "2024-08-06T20:20:35.753066470Z"
     },
     {
       "name": "projects/hc-terraform-testing/locations/us-central1/keyRings/mau-latest-test/cryptoKeys/mau-crypto-key-test/cryptoKeyVersions/4",
       "state": "ENABLED",
       "createTime": "2024-08-06T20:20:38.344795186Z",
       "protectionLevel": "SOFTWARE",
       "algorithm": "EC_SIGN_P256_SHA256",
       "generateTime": "2024-08-06T20:20:38.376814820Z"
     }
   ],
   "totalSize": 4
 }

The output from the tfconfig confirms the latest version by outputting version 4 which can be found in the GET request above:

Outputs:

latest_version = 4
latest_version_crypto_key = "projects/hc-terraform-testing/locations/us-central1/keyRings/mau-latest-test/cryptoKeys/mau-crypto-key-test"
latest_version_name = "projects/hc-terraform-testing/locations/us-central1/keyRings/mau-latest-test/cryptoKeys/mau-crypto-key-test/cryptoKeyVersions/4"

Release Note Template for Downstream PRs (will be copied)

`google_kms_crypto_key_version_latest`

modular-magician · 2024-08-06T22:50:26Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 2 files changed, 221 insertions(+))
google-beta provider: Diff ( 2 files changed, 221 insertions(+))

modular-magician · 2024-08-06T22:56:17Z

Tests analytics

Total tests: 38
Passed tests: 34
Skipped tests: 4
Affected tests: 0

Click here to see the affected service packages

kms

$\textcolor{green}{\textsf{All tests passed!}}$

View the build log

modular-magician · 2024-08-06T23:15:18Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 276 insertions(+))
google-beta provider: Diff ( 3 files changed, 276 insertions(+))

modular-magician · 2024-08-06T23:19:33Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 341 insertions(+))
google-beta provider: Diff ( 4 files changed, 341 insertions(+))

modular-magician · 2024-08-06T23:20:59Z

Tests analytics

Total tests: 39
Passed tests: 34
Skipped tests: 4
Affected tests: 1

Click here to see the affected service packages

kms

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic

Get to know how VCR tests work

modular-magician · 2024-08-06T23:21:15Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

modular-magician · 2024-08-06T23:25:16Z

Tests analytics

Total tests: 39
Passed tests: 34
Skipped tests: 4
Affected tests: 1

Click here to see the affected service packages

kms

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic

Get to know how VCR tests work

modular-magician · 2024-08-06T23:25:30Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

modular-magician · 2024-08-06T23:37:06Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 340 insertions(+))
google-beta provider: Diff ( 4 files changed, 340 insertions(+))

modular-magician · 2024-08-06T23:42:56Z

Tests analytics

Total tests: 39
Passed tests: 34
Skipped tests: 4
Affected tests: 1

Click here to see the affected service packages

kms

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic

Get to know how VCR tests work

modular-magician · 2024-08-06T23:43:10Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

modular-magician · 2024-08-07T00:02:52Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 336 insertions(+))
google-beta provider: Diff ( 4 files changed, 336 insertions(+))

modular-magician · 2024-08-07T00:08:42Z

Tests analytics

Total tests: 39
Passed tests: 34
Skipped tests: 4
Affected tests: 1

Click here to see the affected service packages

kms

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic

Get to know how VCR tests work

modular-magician · 2024-08-07T00:08:54Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

BBBmau · 2024-08-07T01:09:24Z

/gcbrun

modular-magician · 2024-08-07T01:22:28Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 336 insertions(+))
google-beta provider: Diff ( 4 files changed, 336 insertions(+))

modular-magician · 2024-08-07T01:28:09Z

Tests analytics

Total tests: 39
Passed tests: 34
Skipped tests: 4
Affected tests: 1

Click here to see the affected service packages

kms

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic

Get to know how VCR tests work

modular-magician · 2024-08-07T01:28:22Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

BBBmau · 2024-08-07T01:31:30Z

looks like in an attempt to add a new cryptoVersion for this test (in order to check for the latest version in a cryptoKey in stead of just relying on checking for one version) it unfortunately led to the test failing even after reverting the change in the latest commits. Marking it as ready for review to get assistance on this.

ScottSuarez · 2024-08-07T15:56:07Z

@slevenick, I'm out most of this week. Feel free to pass this back to me on Monday if it's not resolved by then. I want to make sure I don't hold this up with my schedule.

slevenick · 2024-08-07T16:21:10Z

What's the use case for this resource? It looks like most resources that take a version can use "latest" as a special key to get the latest version

modular-magician · 2024-08-07T16:23:26Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 336 insertions(+))
google-beta provider: Diff ( 4 files changed, 336 insertions(+))

modular-magician · 2024-08-07T16:29:06Z

Tests analytics

Total tests: 39
Passed tests: 34
Skipped tests: 4
Affected tests: 1

Click here to see the affected service packages

kms

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic

Get to know how VCR tests work

modular-magician · 2024-08-07T16:29:19Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

BBBmau · 2024-08-07T17:41:44Z

What's the use case for this resource? It looks like most resources that take a version can use "latest" as a special key to get the latest version

@slevenick This specific data source takes in a number for the version field. Their currently is no way to get the latest cryptoKeyVersion without using a separate curl resource.

(base) ┌─(~/Dev/Scratch/latest_version_kms_crypto_version)───────────────────────────────────────────────────────────────────────────────────────────────────────────────(mau@mau-JKDT676NCP:s116)─┐
└─(10:37:48)──> envchain GCLOUD terraform apply                                                                                                                               ──(Wed,Aug07)─┘
╷
│ Error: Incorrect attribute value type
│ 
│   on main.tf line 23, in data "google_kms_crypto_key_version" "latest":
│   23:   version = "latest"
│ 
│ Inappropriate value for attribute "version": a number is required.

slevenick

Test is failing

slevenick · 2024-08-07T18:10:08Z

mmv1/third_party/terraform/services/kms/data_source_google_kms_crypto_key_version_latest.go

+}
+
+func flattenKmsCryptoKeyVersionLatest(versionsList []interface{}, d *schema.ResourceData, config *transport_tpg.Config, cryptoKeyId string) (interface{}, error) {
+	latestVersion := versionsList[len(versionsList)-1].(map[string]interface{})


Does this always work? If the last key has been disabled or deleted does it show up here, or what would be considered "latest"?

And can you add a test for such a case?

When marking versions as Disabled they are still present in the list but are not available through the API, the following error appears which is also the result of the failing vcr test (will be destroyed is treated the same as a disabled state):

tf output:

Error: googleapi: Error 400: projects/hc-terraform-testing/locations/us-central1/keyRings/mau-latest-test/cryptoKeys/mau-crypto-key-test/cryptoKeyVersions/4 is not enabled, current state is: DESTROY_SCHEDULED.

The version list in gcloud console:

So although it's marked as disabled it will still show up as the latest version. I'll look into adding a filter in order to get versions that have the state as enabled.

Upon working on this more, I'm starting to believe that having a data source where it's only purpose is to grab the latest version seems like not the best option especially when we consider that their is currently no existing cryptoKeyVersions.list data source.

This realization comes from the fact that I would also need to implement pageTokens to cover the case of a cryptoKey containing many versions where it requires pages.

It would be best to actually just focus on a plural cryptoKeyVersions data source where a boolean field can be added that grabs you the latest version.

example:

data "google_kms_crypto_key_versions" "plural" { crypto_key = "mau-crypto-latest-test" latest_version = true }

@slevenick let me know your thoughts on this.

Hmmm, google_kms_crypto_key_versions implies that it returns a collection of versions, and latest_version making that only return 1 seems unusual. Maybe the google_kms_crypto_key_versions could have a latest_version field that contains the latest version though.

after a conversation with @SarahFrench through slack, we agreed that the best approach would be to first add support for a plural crypto_key_versions data source.

From their support for a singular crypo_key_version_latest data source can be then worked on where it uses the same logic from the plural data source for getting the latest version from a paginated list.

I'll be closing this and opening a PR for the plural one first. Thanks for your initial feedback on this! @slevenick

slevenick · 2024-08-07T18:12:24Z

mmv1/third_party/terraform/services/kms/data_source_google_kms_crypto_key_version_latest.go

+			return fmt.Errorf("Error setting CryptoKeyVersion public key: %s", err)
+		}
+	}
+	d.SetId(fmt.Sprintf("//cloudkms.googleapis.com/v1/%s/cryptoKeyVersions/%d", d.Get("crypto_key"), d.Get("version")))


Why use this format for the ID? A more common format would not be prefixed with //cloudkms...

This was based off of how it's setup in data_source_google_kms_crypto_key_version.go:

magic-modules/mmv1/third_party/terraform/services/kms/data_source_google_kms_crypto_key_version.go

Lines 148 to 154 in fdf0c08

if err := d.Set("public_key", flattenKmsCryptoKeyVersionPublicKey(res, d)); err != nil {

return fmt.Errorf("Error setting CryptoKeyVersion public key: %s", err)

}

}

d.SetId(fmt.Sprintf("//cloudkms.googleapis.com/v1/%s/cryptoKeyVersions/%d", d.Get("crypto_key"), d.Get("version")))

return nil

modular-magician · 2024-08-08T03:48:43Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 336 insertions(+))
google-beta provider: Diff ( 4 files changed, 336 insertions(+))

modular-magician · 2024-08-08T03:54:37Z

Tests analytics

Total tests: 39
Passed tests: 34
Skipped tests: 4
Affected tests: 1

Click here to see the affected service packages

kms

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic

Get to know how VCR tests work

modular-magician · 2024-08-08T03:54:51Z

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccDataSourceGoogleKmsCryptoKeyVersionLatest_basic[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

add data_source_kms_crypto_key_version_latest

4d5409d

modular-magician added the service/cloudkms label Aug 6, 2024

BBBmau added 2 commits August 6, 2024 16:02

add test file

d4513e5

add docs

b22761b

BBBmau changed the title ~~add data_google_kms_crypto_key_version_latest~~ add google_kms_crypto_key_version_latest Aug 6, 2024

remove state from test

e45c25f

set version to 3

7d07a58

BBBmau marked this pull request as ready for review August 7, 2024 01:31

BBBmau requested a review from ScottSuarez August 7, 2024 01:39

ScottSuarez requested review from a team and removed request for ScottSuarez August 7, 2024 15:54

ScottSuarez requested review from slevenick and removed request for a team August 7, 2024 15:55

remove extra sprintf argument

0de1236

slevenick requested changes Aug 7, 2024

View reviewed changes

add filter to only grab versions that have the state as ENABLED

058782e

BBBmau marked this pull request as draft August 8, 2024 03:50

BBBmau closed this Aug 9, 2024

BBBmau mentioned this pull request Aug 15, 2024

KMS: add google_kms_crypto_key_versions data source #11455

Merged

This was referenced Aug 23, 2024

KMS: add google_kms_crypto_key_versions data source hashicorp/terraform-provider-google-beta#8026

Merged

KMS: add google_kms_crypto_key_versions data source hashicorp/terraform-provider-google#19241

Merged

	if err := d.Set("public_key", flattenKmsCryptoKeyVersionPublicKey(res, d)); err != nil {
	return fmt.Errorf("Error setting CryptoKeyVersion public key: %s", err)
	}
	}
	d.SetId(fmt.Sprintf("//cloudkms.googleapis.com/v1/%s/cryptoKeyVersions/%d", d.Get("crypto_key"), d.Get("version")))

	return nil

add google_kms_crypto_key_version_latest #11381

add google_kms_crypto_key_version_latest #11381

Conversation

BBBmau commented Aug 6, 2024 • edited Loading

modular-magician commented Aug 6, 2024

Diff report

modular-magician commented Aug 6, 2024

Tests analytics

modular-magician commented Aug 6, 2024

Diff report

modular-magician commented Aug 6, 2024

Diff report

modular-magician commented Aug 6, 2024

Tests analytics

Action taken

modular-magician commented Aug 6, 2024

modular-magician commented Aug 6, 2024

Tests analytics

Action taken

modular-magician commented Aug 6, 2024

modular-magician commented Aug 6, 2024

Diff report

modular-magician commented Aug 6, 2024

Tests analytics

Action taken

modular-magician commented Aug 6, 2024

modular-magician commented Aug 7, 2024

Diff report

modular-magician commented Aug 7, 2024

Tests analytics

Action taken

modular-magician commented Aug 7, 2024

BBBmau commented Aug 7, 2024

modular-magician commented Aug 7, 2024

Diff report

modular-magician commented Aug 7, 2024

Tests analytics

Action taken

modular-magician commented Aug 7, 2024

BBBmau commented Aug 7, 2024

ScottSuarez commented Aug 7, 2024

slevenick commented Aug 7, 2024

modular-magician commented Aug 7, 2024

Diff report

modular-magician commented Aug 7, 2024

Tests analytics

Action taken

modular-magician commented Aug 7, 2024

BBBmau commented Aug 7, 2024

slevenick left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

modular-magician commented Aug 8, 2024

Diff report

modular-magician commented Aug 8, 2024

Tests analytics

Action taken

modular-magician commented Aug 8, 2024

add `google_kms_crypto_key_version_latest` #11381

add `google_kms_crypto_key_version_latest` #11381

BBBmau commented Aug 6, 2024 •

edited

Loading