Skip to content

Commit 566f948

Browse files
SarahFrenchkumadeerileykarsonmaxi-citBBBmau
authored
Sync main branch into FEATURE-BRANCH-provider-functions (#10032)
* fixes: permadiff issue if event trigger region is not specified (#9989) * fixes: permadiff issue if event trigger region is not specified - fixes hashicorp/terraform-provider-google#17161 * chore: Updated cloudfunction2 test with no explicit trigger_region * Fix GKE front matter, bad field docs (#10018) * Fix GKE front matter, bad field docs * Update container_cluster.html.markdown * added default value for minPortsPerVM field at "google_compute_router_nat" (#9712) * added default value for statis allocation * setting the default from API * add newline removal bash command in guidelines (#9734) * Update hashicorp/terraform-plugin-framework, hashicorp/terraform-plugin-mux, hashicorp/terraform-plugin-sdk/v2 (#10009) * Adding Data Source Forwarding Rules (#10004) * Fix broken terraform datasource google_compute_machine_types example (#10020) * Add check for Environment proto field before accessing in Dataflow provider (#10016) * Add comments to skipped sweeper (#10023) * Add comments for skipped sweepers (#10024) * C3 version schema (#9986) * add support for build number in composerEnvironmentVersionRegexp and composerImageVersionDiffSuppress * make build number optional * regroup regex, cleaner comparison of versions * correction * Add Resource Manager Tags support to 'google_container_cluster' (#9531) * resourceManagerTags added to Cluster Node Config schema * update beta tag * add cluster and node proto tests * add expand and flatten proto * removed beta tag * added to documentation * added resource manager tags to auto pilot * migrating resourceManagerTags tests * migrating node_pools test * migrating additional tests * minor fixes * fixing tests * add in-place update support * fixed tests * fixed annotations * validated clusters and node pools tests. Isolated node pool auto config * isolated resource manager tags from docs * fixed permission issue * fixed spaces * fixed non determinism on tag keys * removed auto_pilot rmts * fixed time_sleep * add depends_on to IAM policies * Add volume replication support for Google Cloud NetApp Volumes (#9816) * Initial replication commit * Cleanup work - Renamed a lot of files to make clear which resource the belong to - Updated documentation for resource fields - Renamed a few resource fields and changed some types - Disabled the custom code for now. Needs to be discussed first * Update example file * Updated example file * Major updates - Reorganisation of block - Reorganisation of fields to match API documentation - Updated example parameters - Added missing API fields - Improved descriptions - * For replication deletion, stop replication first * Add support for deleting destination volume on replication delete * Make volumes deletable in presence of snapshots. This change will be PRed for volume resource independently. Adding it here while it is not in main. * Improving debug error message * yaml check and format fix * Add wait for mirror to initialize. Required to run destroy shortly after create. * Wait on destroy, not on create * Make deleting a replication more robust - doc improvements - started to implement stop/resume. More work required. - renamed a few files to better reflect what they are good for * adding support for stop/resume * yamlformat and lint * Add force delete to delete volumes with nested snapshots * resource test first version * More changes to make tests solid - Introduced new parameter to wait for mirror_status==MIRRORED - more mirror state reconciliation * Test updates * few cleanups * Make virtual field verifies happy * Minor test improvements * More fine tuning - Remove merge conflict in volume.yaml - make generated test work - make output field work - ignore_read for virtual fields * Resource name change as suggested by @slevenick * Remove snapshot code block and fix typo * Detect manual stop/resume actions * Remove ignore_read for deletion_policy * - Made destinationVolumeParameters immutable. It still requires ignore_read. - removed ignore_read from virtual_fields * destinationVolumeParameters are only evaluated at create. Make the immutable. * Name cleanups and comment improvements * removed comment Co-authored-by: Shuya Ma <[email protected]> * tabs to spaces in resource block Co-authored-by: Shuya Ma <[email protected]> * Updates to address review comments - make wait_for_mirror also work for stop/resume, additionally to create - convert tabs in test resource blocks to spaces - fix typos * Rewording of comments Co-authored-by: Shuya Ma <[email protected]> --------- Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> * Ensured that beta runs in TeamCity use only beta paths (#10025) * Ensured that beta runs in TeamCity use only beta paths * Added tests for sweeper package path --------- Co-authored-by: Deepak Kumar <[email protected]> Co-authored-by: Riley Karson <[email protected]> Co-authored-by: Max Portocarrero CI&T <[email protected]> Co-authored-by: Mauricio Alvarez Leon <[email protected]> Co-authored-by: Jesse Liddle <[email protected]> Co-authored-by: Wessel Blokzijl <[email protected]> Co-authored-by: Jack McCluskey <[email protected]> Co-authored-by: hao-nan-li <[email protected]> Co-authored-by: Salome Papiashvili <[email protected]> Co-authored-by: Oliver Krause <[email protected]> Co-authored-by: G-NamanGupta <[email protected]> Co-authored-by: Shuya Ma <[email protected]> Co-authored-by: Stephen Lewis (Burrows) <[email protected]>
1 parent a9f2961 commit 566f948

40 files changed

+2002
-226
lines changed

mmv1/products/alloydb/User.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ references: !ruby/object:Api::Resource::ReferenceLinks
2424
'AlloyDB': 'https://cloud.google.com/alloydb/docs/'
2525
api: 'https://cloud.google.com/alloydb/docs/reference/rest/v1/projects.locations.clusters.users/create'
2626
import_format: ['projects/{{project}}/locations/{{location}}/clusters/{{cluster}}/users/{{user_id}}']
27+
# Skipping the sweeper because instances will be deleted during cluster sweeps
2728
skip_sweeper: true
2829
autogen_async: true
2930
custom_code: !ruby/object:Provider::Terraform::CustomCode

mmv1/products/cloudfunctions2/Function.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -594,6 +594,7 @@ properties:
594594
events originating in this region. It can be the same
595595
region as the function, a different region or multi-region, or the global
596596
region. If not provided, defaults to the same region as the function.
597+
default_from_api: true
597598
- !ruby/object:Api::Type::String
598599
name: 'eventType'
599600
description: 'Required. The type of event to observe.'

mmv1/products/compute/RouterNat.yaml

+2-1
Original file line numberDiff line numberDiff line change
@@ -236,7 +236,8 @@ properties:
236236
- !ruby/object:Api::Type::Integer
237237
name: minPortsPerVm
238238
description: |
239-
Minimum number of ports allocated to a VM from this NAT.
239+
Minimum number of ports allocated to a VM from this NAT. Defaults to 64 for static port allocation and 32 dynamic port allocation if not set.
240+
default_from_api: true
240241
- !ruby/object:Api::Type::Integer
241242
name: maxPortsPerVm
242243
description: |

mmv1/products/firebase/AndroidApp.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ import_format:
5252
'{{app_id}}',
5353
]
5454
autogen_async: true
55+
# Sweeper skipped as this resource has customized deletion.
5556
skip_sweeper: true
5657
identity:
5758
- appId

mmv1/products/firebase/AppleApp.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ import_format:
5252
'{{app_id}}'
5353
]
5454
autogen_async: true
55+
# Sweeper skipped as this resource has customized deletion.
5556
skip_sweeper: true
5657
identity:
5758
- appId

mmv1/products/firebase/WebApp.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ import_format:
5252
'{{app_id}}',
5353
]
5454
autogen_async: true
55+
# Sweeper skipped as this resource has customized deletion.
5556
skip_sweeper: true
5657
identity:
5758
- appId
+289
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,289 @@
1+
# Copyright 2023 Google Inc.
2+
# Licensed under the Apache License, Version 2.0 (the "License");
3+
# you may not use this file except in compliance with the License.
4+
# You may obtain a copy of the License at
5+
#
6+
# http://www.apache.org/licenses/LICENSE-2.0
7+
#
8+
# Unless required by applicable law or agreed to in writing, software
9+
# distributed under the License is distributed on an "AS IS" BASIS,
10+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
# See the License for the specific language governing permissions and
12+
# limitations under the License.
13+
14+
---
15+
!ruby/object:Api::Resource
16+
name: "VolumeReplication"
17+
description: |
18+
Volume replication creates an asynchronous mirror of a volume in a different location. This capability
19+
lets you use the replicated volume for critical application activity in case of a location-wide outage
20+
or disaster.
21+
22+
A new destination volume is created as part of the replication resource. It's content is updated on a
23+
schedule with content of the source volume. It can be used as a read-only copy while the mirror is
24+
enabled, or as an independent read-write volume while the mirror is stopped. A destination volume will
25+
also contain the snapshots of the source volume. Resuming a mirror will overwrite all changes on the
26+
destination volume with the content of the source volume. While is mirror is enabled, all configuration
27+
changes done to source or destination volumes are automatically done to both. Please note that the
28+
destination volume is not a resource managed by Terraform.
29+
30+
Reversing the replication direction is not supported through the provider.
31+
references: !ruby/object:Api::Resource::ReferenceLinks
32+
guides:
33+
"Documentation": "https://cloud.google.com/netapp/volumes/docs/protect-data/about-volume-replication"
34+
api: "https://cloud.google.com/netapp/volumes/docs/reference/rest/v1/projects.locations.volumes.replications"
35+
base_url: projects/{{project}}/locations/{{location}}/volumes/{{volume_name}}/replications
36+
self_link: projects/{{project}}/locations/{{location}}/volumes/{{volume_name}}/replications/{{name}}
37+
create_url: projects/{{project}}/locations/{{location}}/volumes/{{volume_name}}/replications?replicationId={{name}}
38+
update_url: projects/{{project}}/locations/{{location}}/volumes/{{volume_name}}/replications/{{name}}
39+
update_verb: :PATCH
40+
update_mask: true
41+
autogen_async: true
42+
async: !ruby/object:Api::OpAsync
43+
operation: !ruby/object:Api::OpAsync::Operation
44+
base_url: "{{op_id}}"
45+
id_format: "projects/{{project}}/locations/{{location}}/volumes/{{volume_name}}/replications/{{name}}"
46+
import_format:
47+
[
48+
"projects/{{project}}/locations/{{location}}/volumes/{{volume_name}}/replications/{{name}}",
49+
]
50+
examples:
51+
- !ruby/object:Provider::Terraform::Examples
52+
name: "netapp_volume_replication_create"
53+
primary_resource_id: "test_replication"
54+
vars:
55+
source_pool_name: "source-pool"
56+
destination_pool_name: "destination-pool"
57+
volume_name: "source-volume"
58+
replication_name: "test-replication"
59+
destination_volume: "destination-volume"
60+
network_name: "test-network"
61+
ignore_read_extra:
62+
- "delete_destination_volume"
63+
- "replication_enabled"
64+
- "force_stopping"
65+
- "wait_for_mirror"
66+
test_vars_overrides:
67+
network_name: 'acctest.BootstrapSharedServiceNetworkingConnection(t, "gcnv-network-config-1", acctest.ServiceNetworkWithParentService("netapp.servicenetworking.goog"))'
68+
parameters:
69+
- !ruby/object:Api::Type::String
70+
name: "location"
71+
required: true
72+
immutable: true
73+
url_param_only: true
74+
description: |
75+
Name of region for this resource. The resource needs to be created in the region of the destination volume.
76+
- !ruby/object:Api::Type::String
77+
name: "volumeName"
78+
description: The name of the existing source volume.
79+
required: true
80+
immutable: true
81+
url_param_only: true
82+
- !ruby/object:Api::Type::String
83+
name: "name"
84+
description: The name of the replication. Needs to be unique per location.
85+
required: true
86+
immutable: true
87+
url_param_only: true
88+
properties:
89+
- !ruby/object:Api::Type::Enum
90+
name: "state"
91+
description: |
92+
Indicates the state of replication resource. State of the mirror itself is indicated in mirrorState.
93+
values:
94+
- :STATE_UNSPECIFIED
95+
- :CREATING
96+
- :READY
97+
- :UPDATING
98+
- :DELETING
99+
- :ERROR
100+
output: true
101+
- !ruby/object:Api::Type::String
102+
name: "stateDetails"
103+
description: |
104+
State details of the replication resource.
105+
output: true
106+
- !ruby/object:Api::Type::Enum
107+
name: "role"
108+
description: |
109+
Reverting a replication can swap source and destination volume roles. This field indicates if the `location` hosts
110+
the source or destination volume. For resume and revert and resume operations it is critical to understand
111+
which volume is the source volume, since it will overwrite changes done to the destination volume.
112+
values:
113+
- :REPLICATION_ROLE_UNSPECIFIED
114+
- :SOURCE
115+
- :DESTINATION
116+
output: true
117+
- !ruby/object:Api::Type::Enum
118+
name: "replicationSchedule"
119+
description: |
120+
Specifies the replication interval.
121+
values:
122+
- :EVERY_10_MINUTES
123+
- :HOURLY
124+
- :DAILY
125+
required: true
126+
- !ruby/object:Api::Type::Enum
127+
name: "mirrorState"
128+
description: |
129+
Indicates the state of the mirror between source and destination volumes. Depending on the amount of data
130+
in your source volume, PREPARING phase can take hours or days. mirrorState = MIRRORED indicates your baseline
131+
transfer ended and destination volume became accessible read-only. TRANSFERRING means a MIRRORED volume
132+
currently receives an update. Updated every 5 minutes.
133+
values:
134+
- :MIRROR_STATE_UNSPECIFIED
135+
- :PREPARING
136+
- :MIRRORED
137+
- :STOPPED
138+
- :TRANSFERRING
139+
custom_flatten: templates/terraform/custom_flatten/netapp_volume_replicaton_mirror_state.go.erb
140+
output: true
141+
- !ruby/object:Api::Type::String
142+
name: "createTime"
143+
description: |
144+
Create time of the active directory. A timestamp in RFC3339 UTC "Zulu" format. Examples: "2023-06-22T09:13:01.617Z".
145+
output: true
146+
- !ruby/object:Api::Type::String
147+
name: "destinationVolume"
148+
description: |
149+
Full resource name of destination volume with format: `projects/{{project}}/locations/{{location}}/volumes/{{volumeId}}`
150+
output: true
151+
- !ruby/object:Api::Type::NestedObject
152+
name: "transferStats"
153+
description: |-
154+
Replication transfer statistics. All statistics are updated every 5 minutes.
155+
output: true
156+
properties:
157+
- !ruby/object:Api::Type::String
158+
name: "transferBytes"
159+
description: |
160+
Number of bytes transferred so far in current transfer.
161+
output: true
162+
- !ruby/object:Api::Type::String
163+
name: "totalTransferDuration"
164+
description: |
165+
Total time taken so far during current transfer.
166+
output: true
167+
- !ruby/object:Api::Type::String
168+
name: "lastTransferBytes"
169+
description: |
170+
Size of last completed transfer in bytes.
171+
output: true
172+
- !ruby/object:Api::Type::String
173+
name: "lastTransferDuration"
174+
description: |
175+
Time taken during last completed transfer.
176+
output: true
177+
- !ruby/object:Api::Type::String
178+
name: "lagDuration"
179+
description: |
180+
The elapsed time since the creation of the snapshot on the source volume that was last replicated
181+
to the destination volume. Lag time represents the difference in age of the destination volume
182+
data in relation to the source volume data.
183+
output: true
184+
- !ruby/object:Api::Type::String
185+
name: "updateTime"
186+
description: |
187+
Time when progress was updated last. A timestamp in RFC3339 UTC "Zulu" format. Examples: "2023-06-22T09:13:01.617Z".
188+
output: true
189+
- !ruby/object:Api::Type::String
190+
name: "lastTransferEndTime"
191+
description: |
192+
Time when last transfer completed. A timestamp in RFC3339 UTC "Zulu" format. Examples: "2023-06-22T09:13:01.617Z".
193+
output: true
194+
- !ruby/object:Api::Type::String
195+
name: "lastTransferError"
196+
description: |
197+
A message describing the cause of the last transfer failure.
198+
output: true
199+
- !ruby/object:Api::Type::KeyValueLabels
200+
name: "labels"
201+
description: |
202+
Labels as key value pairs. Example: `{ "owner": "Bob", "department": "finance", "purpose": "testing" }`
203+
- !ruby/object:Api::Type::NestedObject
204+
name: "destinationVolumeParameters"
205+
description: |-
206+
Destination volume parameters.
207+
# destinationVolumeParameters is only used on CREATE. Will not be returned on READ.
208+
ignore_read: true
209+
properties:
210+
- !ruby/object:Api::Type::String
211+
name: "storagePool"
212+
description: |
213+
Name of an existing storage pool for the destination volume with format: `projects/{{project}}/locations/{{location}}/storagePools/{{poolId}}`
214+
immutable: true
215+
required: true
216+
- !ruby/object:Api::Type::String
217+
name: "volumeId"
218+
description: |
219+
Name for the destination volume to be created. If not specified, the name of the source volume will be used.
220+
immutable: true
221+
default_from_api: true
222+
- !ruby/object:Api::Type::String
223+
name: "shareName"
224+
description: |
225+
Share name for destination volume. If not specified, name of source volume's share name will be used.
226+
immutable: true
227+
default_from_api: true
228+
- !ruby/object:Api::Type::String
229+
name: "description"
230+
description: |
231+
Description for the destination volume.
232+
immutable: true
233+
- !ruby/object:Api::Type::String
234+
name: "sourceVolume"
235+
description: |
236+
Full resource name of source volume with format: `projects/{{project}}/locations/{{location}}/volumes/{{volumeId}}`
237+
output: true
238+
- !ruby/object:Api::Type::Boolean
239+
name: "healthy"
240+
description: |
241+
Condition of the relationship. Can be one of the following:
242+
- true: The replication relationship is healthy. It has not missed the most recent scheduled transfer.
243+
- false: The replication relationship is not healthy. It has missed the most recent scheduled transfer.
244+
output: true
245+
- !ruby/object:Api::Type::String
246+
name: "description"
247+
description: |
248+
An description of this resource.
249+
virtual_fields:
250+
- !ruby/object:Api::Type::Boolean
251+
name: "delete_destination_volume"
252+
description: |
253+
A destination volume is created as part of replication creation. The destination volume will not became
254+
under Terraform management unless you import it manually. If you delete the replication, this volume
255+
will remain.
256+
Setting this parameter to true will delete the *current* destination volume when destroying the
257+
replication. If you reversed the replication direction, this will be your former source volume!
258+
For production use, it is recommended to keep this parameter false to avoid accidental volume
259+
deletion. Handle with care. Default is false.
260+
default_value: false
261+
- !ruby/object:Api::Type::Boolean
262+
name: "replication_enabled"
263+
description: |
264+
Set to false to stop/break the mirror. Stopping the mirror makes the destination volume read-write
265+
and act independently from the source volume.
266+
Set to true to enable/resume the mirror. WARNING: Resuming a mirror overwrites any changes
267+
done to the destination volume with the content of the source volume.
268+
default_value: true
269+
- !ruby/object:Api::Type::Boolean
270+
name: "force_stopping"
271+
description: |
272+
Only replications with mirror_state=MIRRORED can be stopped. A replication in mirror_state=TRANSFERRING
273+
currently receives an update and stopping the update might be undesirable. Set this parameter to true
274+
to stop anyway. All data transferred to the destination will be discarded and content of destination
275+
volume will remain at the state of the last successful update. Default is false.
276+
default_value: false
277+
- !ruby/object:Api::Type::Boolean
278+
name: "wait_for_mirror"
279+
description: |
280+
Replication resource state is independent of mirror_state. With enough data, it can take many hours
281+
for mirror_state to reach MIRRORED. If you want Terraform to wait for the mirror to finish on
282+
create/stop/resume operations, set this parameter to true. Default is false.
283+
default_value: false
284+
custom_code: !ruby/object:Provider::Terraform::CustomCode
285+
constants: templates/terraform/constants/netapp_volume_replication.go.erb
286+
post_create: templates/terraform/post_create/netapp_volume_replication_post_create.go.erb
287+
pre_delete: templates/terraform/pre_delete/netapp_volume_replication_stop_before_delete.go.erb
288+
post_delete: templates/terraform/post_delete/netapp_volume_replication_delete_destination_volume.go.erb
289+
post_update: templates/terraform/post_update/netapp_volume_replication_mirror_state.go.erb
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
// Custom function to wait for mirrorState target states
2+
func NetAppVolumeReplicationWaitForMirror(d *schema.ResourceData, meta interface{}, targetState string) error {
3+
config := meta.(*transport_tpg.Config)
4+
userAgent, err := tpgresource.GenerateUserAgentString(d, config.UserAgent)
5+
if err != nil {
6+
return err
7+
}
8+
9+
url, err := tpgresource.ReplaceVars(d, config, "{{NetappBasePath}}projects/{{project}}/locations/{{location}}/volumes/{{volume_name}}/replications/{{name}}")
10+
if err != nil {
11+
return err
12+
}
13+
14+
billingProject := ""
15+
16+
project, err := tpgresource.GetProject(d, config)
17+
if err != nil {
18+
return fmt.Errorf("Error fetching project for volume replication: %s", err)
19+
}
20+
billingProject = project
21+
22+
// err == nil indicates that the billing_project value was found
23+
if bp, err := tpgresource.GetBillingProject(d, config); err == nil {
24+
billingProject = bp
25+
}
26+
27+
for {
28+
res, err := transport_tpg.SendRequest(transport_tpg.SendRequestOptions{
29+
Config: config,
30+
Method: "GET",
31+
Project: billingProject,
32+
RawURL: url,
33+
UserAgent: userAgent,
34+
})
35+
if err != nil {
36+
return transport_tpg.HandleNotFoundError(err, d, fmt.Sprintf("NetappVolumeReplication %q", d.Id()))
37+
}
38+
39+
log.Printf("[DEBUG] waiting for mirrorState. actual: %v, target: %v", res["mirrorState"], targetState)
40+
41+
if res["mirrorState"] == targetState {
42+
break
43+
}
44+
45+
time.Sleep(30 * time.Second)
46+
// This method can potentially run for days, e.g. when setting up a replication for a source volume
47+
// with dozens of TiB of data. Timeout handling yes/no?
48+
}
49+
50+
return nil
51+
}

0 commit comments

Comments
 (0)