Add support for Stored Procedure for Apache Spark

[marblejenka]

PR for hashicorp/terraform-provider-google#16953, adding stored procedure for apache spark on google_bigquery_routine resource.

Fixes hashicorp/terraform-provider-google#16953

Release Note Template for Downstream PRs (will be copied)

bigquery: add `spark_options` field to `google_bigquery_routine` resource

[modular-magician]

Hello! I am a robot. It looks like you are a: Community Contributor Googler Core Contributor. Tests will run automatically.

@roaks3, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 571 insertions(+), 3 deletions(-))
Terraform Beta: Diff ( 3 files changed, 571 insertions(+), 3 deletions(-))
TF Conversion: Diff ( 1 file changed, 135 insertions(+))
TF OiCS: Diff ( 8 files changed, 271 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_bigquery_routine (9 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_bigquery_routine" "primary" {
  spark_options {
    archive_uris    = # value needed
    container_image = # value needed
    file_uris       = # value needed
    main_file_uri   = # value needed
  }
}

[modular-magician]

Tests analytics

Total tests: 102
Passed tests 92
Skipped tests: 8
Affected tests: 2

Click here to see the affected service packages

• bigquery

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccBigQueryRoutine_bigQueryRoutineSparkJarExample|TestAccBigQueryRoutine_bigQueryRoutinePysparkExample

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccBigQueryRoutine_bigQueryRoutineSparkJarExample[Debug log]
TestAccBigQueryRoutine_bigQueryRoutinePysparkExample[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

[roaks3]

Overall LGTM and it looks like tests passed. Just a few minor comments

[marblejenka]

@roaks3 Thank you for your feedback. I have made some corrections. I have also commented on a consideration, so I would be happy if you could take a look at that as well.

[ScottSuarez]

/gcbrun

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 4 files changed, 753 insertions(+), 3 deletions(-))
Terraform Beta: Diff ( 4 files changed, 753 insertions(+), 3 deletions(-))
TF Conversion: Diff ( 1 file changed, 135 insertions(+))
TF OiCS: Diff ( 12 files changed, 397 insertions(+))

[modular-magician]

Tests analytics

Total tests: 104
Passed tests 92
Skipped tests: 8
Affected tests: 4

Click here to see the affected service packages

• bigquery

Action taken

Found 4 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccBigQueryRoutine_bigQueryRoutinePysparkMainfileExample|TestAccBigQueryRoutine_bigQueryRoutinePysparkExample|TestAccBigQueryRoutine_bigQueryRoutineSparkJar_Update|TestAccBigQueryRoutine_bigQueryRoutineSparkJarExample

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccBigQueryRoutine_bigQueryRoutinePysparkMainfileExample[Debug log]
TestAccBigQueryRoutine_bigQueryRoutinePysparkExample[Debug log]
TestAccBigQueryRoutine_bigQueryRoutineSparkJar_Update[Debug log]
TestAccBigQueryRoutine_bigQueryRoutineSparkJarExample[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

[roaks3]

LGTM, one remaining comment on the update test to make sure these fields are configured properly.

[roaks3]

I would consider changing main_class and jar_uris with this test, maybe even connection, if that is possible. Note that any fields that cannot be updated in-place should have immutable: true, to make sure that they signal the resource needs to be recreated.

[marblejenka]

Thank you. I will modify some tests. Also, I will send another PR for the definition_body.
There will be a delay in response due to a business trip, but I will send out a PR within this week.

[marblejenka]

/gcbrun

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 4 files changed, 753 insertions(+), 3 deletions(-))
Terraform Beta: Diff ( 4 files changed, 753 insertions(+), 3 deletions(-))
TF Conversion: Diff ( 1 file changed, 135 insertions(+))
TF OiCS: Diff ( 12 files changed, 397 insertions(+))

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 4 files changed, 753 insertions(+), 3 deletions(-))
Terraform Beta: Diff ( 4 files changed, 753 insertions(+), 3 deletions(-))
TF Conversion: Diff ( 1 file changed, 135 insertions(+))
TF OiCS: Diff ( 12 files changed, 397 insertions(+))

[modular-magician]

Tests analytics

Total tests: 107
Passed tests 98
Skipped tests: 8
Affected tests: 1

Click here to see the affected service packages

• bigquery

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccBigQueryRoutine_bigQueryRoutineSparkJar_Update

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccBigQueryRoutine_bigQueryRoutineSparkJar_Update[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

[modular-magician]

Tests analytics

Total tests: 107
Passed tests 98
Skipped tests: 8
Affected tests: 1

Click here to see the affected service packages

• bigquery

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccBigQueryRoutine_bigQueryRoutineSparkJar_Update

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccBigQueryRoutine_bigQueryRoutineSparkJar_Update[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

[marblejenka]

@roaks3 I have updated the test for updates. I believe now we can merge the PR.