Support creating Database Migration Service connection profiles connected to existing Cloud SQL and AlloyDB instances

[owaismawasy]

Support creating DMS connection profiles that link to existing Cloud SQL instances/AlloyDB clusters.

This will allow using such connection profiles as destination to the following migration types:

• PostgreSQL to Cloud SQL (PostgreSQL)
• MySQL to Cloud SQL (MySQL)
• PostgreSQL to AllloyDB

Release Note Template for Downstream PRs (will be copied)

Support creating DMS connection profiles that link to existing Cloud SQL instances/AlloyDB clusters. 

[github-actions]

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

@BBBmau, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 8 files changed, 2164 insertions(+), 55 deletions(-))
google-beta provider: Diff ( 8 files changed, 2164 insertions(+), 55 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 11 insertions(+))
Open in Cloud Shell: Diff ( 12 files changed, 408 insertions(+))

[modular-magician]

Tests analytics

Total tests: 8
Passed tests: 5
Skipped tests: 0
Affected tests: 3

Click here to see the affected service packages

• databasemigrationservice

Action taken

Found 3 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingAlloydbExample
• TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingMysqlExample
• TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingPostgresExample

Get to know how VCR tests work

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 416 insertions(+), 53 deletions(-))
google-beta provider: Diff ( 3 files changed, 416 insertions(+), 53 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 11 insertions(+))
Open in Cloud Shell: Diff ( 12 files changed, 408 insertions(+))

[modular-magician]

Tests analytics

Total tests: 8
Passed tests: 5
Skipped tests: 0
Affected tests: 3

Click here to see the affected service packages

• databasemigrationservice

Action taken

Found 3 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingAlloydbExample
• TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingMysqlExample
• TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingPostgresExample

Get to know how VCR tests work

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 416 insertions(+), 53 deletions(-))
google-beta provider: Diff ( 3 files changed, 416 insertions(+), 53 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 11 insertions(+))
Open in Cloud Shell: Diff ( 12 files changed, 408 insertions(+))

[modular-magician]

Tests analytics

Total tests: 8
Passed tests: 5
Skipped tests: 0
Affected tests: 3

Click here to see the affected service packages

• databasemigrationservice

Action taken

Found 3 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingAlloydbExample
• TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingMysqlExample
• TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingPostgresExample

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingAlloydbExample[Debug log]
TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingMysqlExample[Debug log]
TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingPostgresExample[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$

View the build log or the debug log for each test

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingAlloydbExample[Debug log]
TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingMysqlExample[Debug log]
TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingPostgresExample[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$

View the build log or the debug log for each test

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingAlloydbExample[Debug log]
TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingMysqlExample[Debug log]
TestAccDatabaseMigrationServiceConnectionProfile_databaseMigrationServiceConnectionProfileExistingPostgresExample[Debug log]

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$

View the build log or the debug log for each test

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 416 insertions(+), 53 deletions(-))
google-beta provider: Diff ( 3 files changed, 416 insertions(+), 53 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 11 insertions(+))
Open in Cloud Shell: Diff ( 12 files changed, 408 insertions(+))

[modular-magician]

Tests analytics

Total tests: 8
Passed tests: 8
Skipped tests: 0
Affected tests: 0

Click here to see the affected service packages

• databasemigrationservice

$\textcolor{green}{\textsf{All tests passed!}}$

View the build log

[github-actions]

@BBBmau This PR has been waiting for review for 3 weekdays. Please take a look! Use the label disable-review-reminders to disable these notifications.

[BBBmau]

what's the reasoning behind replacing required with required_with? Is it necessary when the fields are already marked as required?

[owaismawasy]

DMS started to support a new path, called existing-instance, for the following homogenous migration types:

1. MySQL to Cloud SQL (MySQL)
2. PostgreSQL to Cloud SQL (PostgreSQL)
3. PostgreSQL to AlloyDB

With this new existing-instance path, users can create the destination database instance outside of DMS and then create DMS connection profiles that point to these destination instances. In that case, users need to provide only the CloudSQL ID or AlloyDB ID, and they don't need to provide additional connection params like username, password, host, etc.

For example, a destination connection profile for MySQL -> MySQL migrations will look like this:

resource "google_database_migration_service_connection_profile" "source" {
  location              = "us-central1"
  …
  mysql {
    cloud_sql_id = "<csql_id>"
  }
}

This means that connection params (e.g. username, password, etc.) are no longer required in all cases.

That said, for other migration types (e.g. Oracle -> PostgreSQL), it doesn't make sense to provide partial connection params, so we're configuring the resource so that if one connection param is provided (e.g. username), we'll need other params to be provided as well, and we're using the required_with to achieve that.

[BBBmau]

Thank you for the thorough explanation! I also came across this which matches what you just explained: https://cloud.google.com/database-migration/docs/mysql/configure-source-database#overview-mysql