Support to force delete non-current object versions in storage bucket

[anubbhavm]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

We were testing out the force_destroy feature on the google cloud storage bucket resource. According to the documentation, it makes you believe that setting the variable as true should be enough to delete the bucket even if there are objects in the bucket. However, we have noticed that if the bucket has versioning enabled, force_destroy = true is not enough and the destroy fails.

We'd like to request an enhancement so that force_destroy = true destroys the bucket even if it has versioning enabled and some non-current object versions in it.

New or Affected Resource(s)

• google_storage_bucket

b/275624079

[upodroid]

@rileykarson

This already exists in Terraform.

https://cloud.google.com/storage/docs/object-versioning

I ran the TestAccStorageBucket_forceDestroyWithVersioning test and it creates a bucket and 2 objects and one of them is non-current which is what the OP is asking for.

Full log: https://gist.github.com/upodroid/82817606c264b484fca73f5063bd9515

Relevant snippet:

---[ REQUEST ]---------------------------------------
GET /storage/v1/b/tf-test-acc-bucket-4967628847734850547/o?alt=json&prettyPrint=false&versions=true HTTP/1.1
Host: storage.googleapis.com
User-Agent: google-api-go-client/0.5 Terraform/0.13.2 (+https://www.terraform.io) Terraform-Plugin-SDK/2.4.2 terraform-provider-google-beta/acc
X-Goog-Api-Client: gl-go/1.15.5 gdcl/20210128
Accept-Encoding: gzip


-----------------------------------------------------
2021/02/24 18:12:27 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 200 OK
Content-Length: 1721
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Wed, 24 Feb 2021 18:12:27 GMT
Expires: Wed, 24 Feb 2021 18:12:27 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ABg5-Uzhj4h_VarCCn0D5kbr7wgHMxtUzWNay1syse7XUJgvbWYBkFX9O_IupavbvTdMXlHtMRLuwhXEfVG7QBUPk6A

{
 "kind": "storage#objects",
 "items": [
  {
   "kind": "storage#object",
   "id": "tf-test-acc-bucket-4967628847734850547/bucketDestroyTestFile/1614190276681407",
   "selfLink": "https://www.googleapis.com/storage/v1/b/tf-test-acc-bucket-4967628847734850547/o/bucketDestroyTestFile",
   "mediaLink": "https://storage.googleapis.com/download/storage/v1/b/tf-test-acc-bucket-4967628847734850547/o/bucketDestroyTestFile?generation=1614190276681407&alt=media",
   "name": "bucketDestroyTestFile",
   "bucket": "tf-test-acc-bucket-4967628847734850547",
   "generation": "1614190276681407",
   "metageneration": "1",
   "contentType": "text/plain; charset=utf-8",
   "storageClass": "STANDARD",
   "size": "4",
   "md5Hash": "CY9rzUYh03PK3k6DJie09g==",
   "crc32c": "hqBywA==",
   "etag": "CL+9qrCPg+8CEAE=",
   "timeCreated": "2021-02-24T18:11:16.709Z",
   "updated": "2021-02-24T18:11:16.709Z",
   "timeDeleted": "2021-02-24T18:12:01.112Z",
   "timeStorageClassUpdated": "2021-02-24T18:11:16.709Z"
  },
  {
   "kind": "storage#object",
   "id": "tf-test-acc-bucket-4967628847734850547/bucketDestroyTestFile/1614190321003623",
   "selfLink": "https://www.googleapis.com/storage/v1/b/tf-test-acc-bucket-4967628847734850547/o/bucketDestroyTestFile",
   "mediaLink": "https://storage.googleapis.com/download/storage/v1/b/tf-test-acc-bucket-4967628847734850547/o/bucketDestroyTestFile?generation=1614190321003623&alt=media",
   "name": "bucketDestroyTestFile",
   "bucket": "tf-test-acc-bucket-4967628847734850547",
   "generation": "1614190321003623",
   "metageneration": "1",
   "contentType": "text/plain; charset=utf-8",
   "storageClass": "STANDARD",
   "size": "4",
   "md5Hash": "CY9rzUYh03PK3k6DJie09g==",
   "crc32c": "hqBywA==",
   "etag": "COfYu8WPg+8CEAE=",
   "timeCreated": "2021-02-24T18:12:01.112Z",
   "updated": "2021-02-24T18:12:01.112Z",
   "timeStorageClassUpdated": "2021-02-24T18:12:01.112Z"
  }
 ]
}
-----------------------------------------------------
2021/02/24 18:12:27 [DEBUG] Retry Transport: Stopping retries, last request was successful
2021/02/24 18:12:27 [DEBUG] Retry Transport: Returning after 1 attempts
2021/02/24 18:12:27 [DEBUG] GCS Bucket attempting to forceDestroy

2021/02/24 18:12:27 [DEBUG] Found bucketDestroyTestFile
2021/02/24 18:12:27 [DEBUG] Found bucketDestroyTestFile
2021/02/24 18:12:27 [INFO] Instantiating Google Storage client for path https://storage.googleapis.com/storage/v1/
2021/02/24 18:12:27 [INFO] Instantiating Google Storage client for path https://storage.googleapis.com/storage/v1/
2021/02/24 18:12:27 [DEBUG] Retry Transport: starting RoundTrip retry loop
2021/02/24 18:12:27 [DEBUG] Retry Transport: starting RoundTrip retry loop
2021/02/24 18:12:27 [DEBUG] Retry Transport: request attempt 0
2021/02/24 18:12:27 [DEBUG] Retry Transport: request attempt 0
2021/02/24 18:12:27 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
DELETE /storage/v1/b/tf-test-acc-bucket-4967628847734850547/o/bucketDestroyTestFile?alt=json&generation=1614190276681407&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: google-api-go-client/0.5 Terraform/0.13.2 (+https://www.terraform.io) Terraform-Plugin-SDK/2.4.2 terraform-provider-google-beta/acc
X-Goog-Api-Client: gl-go/1.15.5 gdcl/20210128
Accept-Encoding: gzip


-----------------------------------------------------
2021/02/24 18:12:27 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
DELETE /storage/v1/b/tf-test-acc-bucket-4967628847734850547/o/bucketDestroyTestFile?alt=json&generation=1614190321003623&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: google-api-go-client/0.5 Terraform/0.13.2 (+https://www.terraform.io) Terraform-Plugin-SDK/2.4.2 terraform-provider-google-beta/acc
X-Goog-Api-Client: gl-go/1.15.5 gdcl/20210128
Accept-Encoding: gzip


-----------------------------------------------------
2021/02/24 18:12:28 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 204 No Content
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json
Date: Wed, 24 Feb 2021 18:12:28 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ABg5-Uwv0CnILxM2UYbr2l3Af0r7vfVqtQpbt57L2aHefkWirwapJJVeOmiGuGr01dG68KwDUu1VhjdVDQqcG7bCTqM


-----------------------------------------------------
2021/02/24 18:12:28 [DEBUG] Retry Transport: Stopping retries, last request was successful
2021/02/24 18:12:28 [DEBUG] Retry Transport: Returning after 1 attempts
2021/02/24 18:12:28 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 204 No Content
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json
Date: Wed, 24 Feb 2021 18:12:28 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ABg5-UzyxGzIbVdaVSYxU5jtVhe0Nmi-cWI6iopVjR-bASh_gh5sBVL9fJPud9HL2YzDG-AbErgu8H-MrD8hG1lme_A

[upodroid]

@rileykarson Bump 😄

[rileykarson]

Ah, good catch. @anubbhavm can you share debug logs from a failed attempt to destroy a bucket with versioned items?

[gurusai-voleti]

@anubbhavm how many objects and versions of objects are present in the bucket?