-
Notifications
You must be signed in to change notification settings - Fork 1.8k
BigQuery
: support encryptionConfiguration
in google_bigquery_data_transfer_config
#11478
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 4 commits
1fb1281
44e7607
4bcf27b
c993345
50ff7c7
95e0895
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -46,13 +46,22 @@ examples: | |
vars: | ||
display_name: 'my-query' | ||
dataset_id: 'my_dataset' | ||
- !ruby/object:Provider::Terraform::Examples | ||
name: 'bigquerydatatransfer_config_cmek' | ||
skip_test: true | ||
primary_resource_id: 'query_config_cmek' | ||
vars: | ||
dataset_id: 'example_dataset' | ||
key_name: 'example-key' | ||
keyring_name: 'example-keyring' | ||
- !ruby/object:Provider::Terraform::Examples | ||
skip_test: true | ||
name: 'bigquerydatatransfer_config_salesforce' | ||
primary_resource_id: 'salesforce_config' | ||
vars: | ||
display_name: 'my-salesforce-config' | ||
dataset_id: 'my_dataset' | ||
|
||
parameters: | ||
- !ruby/object:Api::Type::String | ||
name: 'location' | ||
|
@@ -172,6 +181,16 @@ properties: | |
reingests data for [today-10, today-1], rather than ingesting data for | ||
just [today-1]. Only valid if the data source supports the feature. | ||
Set the value to 0 to use the default value. | ||
- !ruby/object:Api::Type::NestedObject | ||
name: 'encryptionConfiguration' | ||
description: | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. To be future-proof, can we make this description more generic and not specific about encryption key? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. updated description to match that found in API Reference. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. According to https://cloud.google.com/bigquery/docs/reference/datatransfer/rest/v1/projects.locations.transferConfigs#encryptionconfiguration - "The name of the KMS key used for encrypting BigQuery data." is the API description of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. apologies seems like i got the descriptions mixed up, should be correct now. |
||
Represents the encryption configuration for a transfer. | ||
properties: | ||
- !ruby/object:Api::Type::String | ||
name: 'kmsKeyName' | ||
required: true | ||
description: | | ||
The name of the KMS key used for encrypting BigQuery data. | ||
- !ruby/object:Api::Type::Boolean | ||
name: 'disabled' | ||
description: | | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
data "google_project" "project" { | ||
} | ||
|
||
resource "google_project_iam_member" "permissions" { | ||
project = data.google_project.project.project_id | ||
role = "roles/iam.serviceAccountTokenCreator" | ||
member = "serviceAccount:service-${data.google_project.project.number}@gcp-sa-bigquerydatatransfer.iam.gserviceaccount.com" | ||
} | ||
|
||
resource "google_bigquery_data_transfer_config" "<%= ctx[:primary_resource_id] %>" { | ||
depends_on = [google_project_iam_member.permissions] | ||
|
||
display_name = "<%= ctx[:vars]['display_name'] %>" | ||
location = "asia-northeast1" | ||
data_source_id = "scheduled_query" | ||
schedule = "first sunday of quarter 00:00" | ||
destination_dataset_id = google_bigquery_dataset.my_dataset.dataset_id | ||
params = { | ||
destination_table_name_template = "my_table" | ||
write_disposition = "WRITE_APPEND" | ||
query = "SELECT name FROM tabl WHERE x = 'y'" | ||
} | ||
|
||
encryption_configuration { | ||
kms_key_name = google_kms_crypto_key.crypto_key.id | ||
} | ||
} | ||
|
||
resource "google_bigquery_dataset" "my_dataset" { | ||
depends_on = [google_project_iam_member.permissions] | ||
|
||
dataset_id = "<%= ctx[:vars]['dataset_id'] %>" | ||
friendly_name = "foo" | ||
description = "bar" | ||
location = "asia-northeast1" | ||
} | ||
|
||
resource "google_kms_crypto_key" "crypto_key" { | ||
name = "<%= ctx[:vars]['key_name'] %>" | ||
key_ring = google_kms_key_ring.key_ring.id | ||
} | ||
|
||
resource "google_kms_key_ring" "key_ring" { | ||
name = "<%= ctx[:vars]['keyring_name'] %>" | ||
location = "us" | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any particular reason to skip the test? If so it looks like the added field is not covered by any test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was based off of another example from
bigquery_dataset
which includes cmek support but doesn't include a test: #2226I believe we skip it due to the test including a cryptoKeyRing creation, which we are unable to remove: https://cloud.google.com/kms/docs/reference/rest/v1/projects.locations.keyRings
I've included a test that includes the
kmsKeyName
field.