[HDInsight] Update api version to 2025-01-15-preview #27837

2258093686 · 2025-05-26T06:28:25Z

Description

Mandatory Checklist

Please choose the target release of Azure PowerShell. (⚠️Target release is a different concept from API readiness. Please click below links for details.)
- General release
- Public preview
- Private preview
- Engineering build
- No need for a release
Check this box to confirm: I have read the Submitting Changes section of CONTRIBUTING.md and reviewed the following information:

SHOULD update ChangeLog.md file(s) appropriately
- Update src/{{SERVICE}}/{{SERVICE}}/ChangeLog.md.
  - A snippet outlining the change(s) made in the PR should be written under the ## Upcoming Release header in the past tense.
- Should not change ChangeLog.md if no new release is required, such as fixing test case only.
SHOULD regenerate markdown help files if there is cmdlet API change. Instruction
SHOULD have proper test coverage for changes in pull request.
SHOULD NOT adjust version of module manually in pull request

…ayEntraUserInfo ; New-AzHDInsightClusterGatewayEntraUserInfo modify cmd: New-AzHDInsightCluster

azure-client-tools-bot-prd · 2025-05-26T06:28:28Z

Thanks for your contribution! The pull request validation has started. Please revisit this comment for updated status.

Copilot

Pull Request Overview

This PR updates the HDInsight API version to 2025-01-15-preview and adapts the cmdlets, SDK, and tests accordingly. Key changes include:

Parameter updates in cmdlets to support the new RestAuthEntraUsers input and making HttpCredential optional.
Updates to test session records and PowerShell scripts to align with the new API version and associated behavior.
Modifications in the SDK and documentation to reference the updated API endpoints.

Reviewed Changes

Copilot reviewed 36 out of 36 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/HDInsight/HDInsight/ManagementCommands/SetAzureHDInsightGatewaySettingsCommand.cs	Adjusted parameter handling and error conditions to support either HttpCredential or RestAuthEntraUsers.
src/HDInsight/HDInsight/ManagementCommands/NewAzureHDInsightClusterGatewayEntraUserInfoCommand.cs	Added a new cmdlet to create Entra user info objects for gateway access.
src/HDInsight/HDInsight/ManagementCommands/NewAzureHDInsightClusterCommand.cs	Made HttpCredential optional and added the RestAuthEntraUsers parameter when invoking the cluster creation helper.
src/HDInsight/HDInsight/ManagementCommands/GetAzureHDInsightClusterGatewayEntraUserInfoCommand.cs	Updated parameter sets to correctly resolve input by name, resource ID, or input object.
src/HDInsight/HDInsight/Constants.cs	Added a new constant key for EntraUsers under GatewayConfigurations.
src/HDInsight/HDInsight/ClusterCreateHelper.cs	Updated the method signature to accept RestAuthEntraUsers and added related parameter validation; left some commented-out legacy code.
Various test and PSD1 files	Updated API version endpoints, cmdlet exports, and test script parameters to reflect the new API version and updated command behavior.

src/HDInsight/HDInsight/ClusterCreateHelper.cs

aim-for-better · 2025-06-06T02:10:18Z

src/HDInsight/HDInsight.Management.Sdk/Generated/HDInsightManagementClient.cs

@@ -310,6 +310,9 @@ public HDInsightManagementClient(System.Uri baseUri, Microsoft.Rest.ServiceClien
        /// <param name='rootHandler'>
        /// Optional. The http client handler used to handle http transport.
        /// </param>
+        /// <param name='handlers'>


Is this generated? why is there a new added handlers param?

aim-for-better · 2025-06-06T02:17:15Z

src/HDInsight/HDInsight.Management.Sdk/Generated/HDInsightManagementClient.cs

@@ -310,6 +310,9 @@ public HDInsightManagementClient(System.Uri baseUri, Microsoft.Rest.ServiceClien
        /// <param name='rootHandler'>
        /// Optional. The http client handler used to handle http transport.
        /// </param>
+        /// <param name='handlers'>


Is this generated? why is there a new added handlers param?

aim-for-better · 2025-06-06T02:22:54Z

src/HDInsight/HDInsight.Management.Sdk/Generated/Models/UpdateGatewaySettingsParameters.cs

@@ -8,7 +8,8 @@ namespace Microsoft.Azure.Management.HDInsight.Models
    using System.Linq;

    /// <summary>
-    /// The update gateway settings request parameters.
+    /// The update gateway settings request parameters. Note either basic or entra
+    /// user should be provided at a time.


aim-for-better · 2025-06-06T02:24:16Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightAutoscaleTests.ps1

@@ -28,7 +28,7 @@ function Test-AutoscaleRelatedCommands{
 		$cluster = New-AzHDInsightCluster -Location $params.location -ResourceGroupName $params.resourceGroupName `
 		-ClusterName $params.clusterName -ClusterSizeInNodes $params.clusterSizeInNodes -ClusterType $params.clusterType `
 		-StorageAccountResourceId $params.storageAccountResourceId -StorageAccountKey $params.storageAccountKey `
-		-HttpCredential $params.httpCredential -SshCredential $params.sshCredential `
+		-HttpCredential $params.httpCredential -SshCredential $params.sshCredential -Version "5.1" `


if without -Version, what is the default version?

aim-for-better · 2025-06-06T02:24:46Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightAzureMonitorTests.ps1

 		# prepare parameter for creating parameter
 		# $params= Prepare-ClusterCreateParameter -location $location

 		# create cluster that will be used throughout test
-		$cluster = Get-AzHDInsightCluster -ResourceGroupName yuchen-ps-test -ClusterName spark51
+		$cluster = Get-AzHDInsightCluster -ResourceGroupName "wut-tip2-esp" -ClusterName "test5-wut-tip2"


change this to group-ps-test

aim-for-better · 2025-06-06T02:24:58Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightAzureMonitorTests.ps1

 		# prepare parameter for creating parameter
 		# $params= Prepare-ClusterCreateParameter -location $location

 		# create cluster that will be used throughout test
-		$cluster = Get-AzHDInsightCluster -ResourceGroupName yuchen-ps-test -ClusterName spark51
+		$cluster = Get-AzHDInsightCluster -ResourceGroupName "wut-tip2-esp" -ClusterName "test5-wut-tip2"


change this

aim-for-better · 2025-06-06T02:25:38Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightAzureMonitorTests.ps1

@@ -102,7 +102,7 @@ function Test-AzureMonitorAgentRelatedCommands{

 		Assert-NotNull $workspaceId
 		Assert-NotNull $primaryKey
-		Enable-AzHDInsightAzureMonitorAgent -ClusterName $cluster.Name -ResourceGroup $cluster.ResourceGroup -WorkspaceId $workspaceId -Primary  $primaryKey
+		Enable-AzHDInsightAzureMonitorAgent -ClusterName $cluster.Name -ResourceGroup $cluster.ResourceGroup -WorkspaceId $workspaceId -PrimaryKey  $primaryKey


did you changed this parameter name?

aim-for-better · 2025-06-06T02:27:12Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightClusterTests.ps1

+	}
+	finally
+	{
+		# Delete cluster and resource group


shall you delete the cluster?

aim-for-better · 2025-06-06T02:27:48Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightClusterTests.ps1

+	finally
+	{
+		# Delete cluster and resource group
+		# Remove-AzResourceGroup -ResourceGroupName $params.resourceGroupName


do you cleanup this after all test cases finished?

aim-for-better · 2025-06-06T02:28:30Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightClusterTests.cs

+
+

change to 1 line?

aim-for-better · 2025-06-06T02:29:07Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightClusterTests.ps1

+
+
+
+


reduce the empty line

aim-for-better · 2025-06-06T02:30:22Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightClusterTests.ps1

+	try{
+		 $params= Prepare-ClusterCreateParameter
+		 $entraUserFullInfo = @(@{ObjectId = "00000000-0000-0000-0000-000000000000"; Upn = "[email protected]"; DisplayName = "DisplayName" },@{ObjectId = "00000000-0000-0000-0000-000000000000"; Upn = "[email protected]"; DisplayName = "DisplayName" })
+		 $entraUserIdentity = "test_test.com#EXT#@microsoft.onmicrosoft.com"	


change this value

aim-for-better · 2025-06-06T02:38:10Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightClusterTests.ps1

+function Test-CreateEntraCluster{
+	try{
+		 $params= Prepare-ClusterCreateParameter
+		 $entraUserFullInfo = @(@{ObjectId = "00000000-0000-0000-0000-000000000000"; Upn = "[email protected]"; DisplayName = "DisplayName" },@{ObjectId = "00000000-0000-0000-0000-000000000000"; Upn = "[email protected]"; DisplayName = "DisplayName" })


change this use like contoso.com

aim-for-better · 2025-06-06T02:40:05Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightClusterTests.ps1

+			StorageContainer                = $params.clusterName
+			StorageAccountKey               = $params.storageAccountKey
+			StorageAccountResourceId        = $params.storageAccountResourceId
+		    # EntraUserFullInfo               = $entraUserFullInfo


remove the unused property definition

aim-for-better · 2025-06-06T02:41:20Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightClusterTests.ps1

 		$subnetName="default"

 		# create cluster
 		$cluster = New-AzHDInsightCluster -Location $params.location -ResourceGroupName $params.resourceGroupName `
-		-ClusterName $params.clusterName -ClusterSizeInNodes $params.clusterSizeInNodes -ClusterType $params.clusterType `
+		-ClusterName $params.clusterName -ClusterSizeInNodes $params.clusterSizeInNodes -ClusterType "Spark" `


don't hard code this, use the params to pass the cluster type

The same with below

aim-for-better · 2025-06-06T02:42:48Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightClusterTests.ps1

@@ -567,8 +651,8 @@ Test Update cluster System Assigned Identity
 function Test-UpdateClusterSystemAssigned{
 	try
 	{
-		$rg="yuchen-ps-test"
-		$clusterName="h1-spark"
+		$rg="yukundemo1"


change this value and check similar issue

aim-for-better · 2025-06-06T02:44:53Z

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightMonitoringTests.ps1

@@ -68,6 +68,6 @@ function Test-MonitoringRelatedCommands{
 	finally
 	{
 		# Delete cluster and resource group
-		Remove-AzResourceGroup -ResourceGroupName $cluster.ResourceGroup
+		# Remove-AzResourceGroup -ResourceGroupName $cluster.ResourceGroup


change this back, and make sure can clean up this rg after tests finished

aim-for-better · 2025-06-06T02:46:49Z

src/HDInsight/HDInsight/ChangeLog.md

+  - Added parameter `-EntraUserIdentity` to cmdlet `New-AzHDInsightCluster` to support creating Entra user clusters using one or more ObjectId or UPNs. Automatically resolves Entra user information.
+  - Added parameter `-EntraUserFullInfo` to cmdlet `New-AzHDInsightCluster` to support creating Entra user clusters using an array of hashtables that include ObjectId, UPN, and DisplayName.
+* Added two parameters `-EntraUserIdentity` and `-EntraUserFullInfo` to cmdlet `Set-AzHDInsightGatewayCredential` 
+  - Added parameter `-EntraUserIdentity` to cmdlet `New-AzHDInsightCluster` to support update Entra user clusters using one or more ObjectId or UPNs. Automatically resolves Entra user information.


wrong cmdlet name here

aim-for-better · 2025-06-06T02:47:31Z

src/HDInsight/HDInsight/ChangeLog.md

+  - Added parameter `-EntraUserFullInfo` to cmdlet `New-AzHDInsightCluster` to support creating Entra user clusters using an array of hashtables that include ObjectId, UPN, and DisplayName.
+* Added two parameters `-EntraUserIdentity` and `-EntraUserFullInfo` to cmdlet `Set-AzHDInsightGatewayCredential` 
+  - Added parameter `-EntraUserIdentity` to cmdlet `New-AzHDInsightCluster` to support update Entra user clusters using one or more ObjectId or UPNs. Automatically resolves Entra user information.
+  - Added parameter `-EntraUserFullInfo` to cmdlet `New-AzHDInsightCluster` to support update Entra user clusters using an array of hashtables that include ObjectId, UPN, and DisplayName.


change this

aim-for-better · 2025-06-06T02:52:06Z

src/HDInsight/HDInsight/ClusterConfigurationUtils.cs

+                    var dict = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
+                    foreach (DictionaryEntry entry in user)
+                    {
+                        dict[entry.Key.ToString() ?? ""] = entry.Value.ToString() ?? "";


refactor this, when the entry.key.ToString() will be null?

aim-for-better · 2025-06-06T02:53:14Z

src/HDInsight/HDInsight/ClusterConfigurationUtils.cs

+                foreach (var userDict in userDicts)
+                {
+                    string objectId = userDict.TryGetValue("ObjectId", out var oid) ? oid : null;
+                    string upn = userDict.TryGetValue("Upn", out var u) ? u : null;


so there you required the key is fixed value, you shall write doc for this requirement when define the entrauserfullinfo

aim-for-better · 2025-06-06T02:54:05Z

src/HDInsight/HDInsight/ClusterConfigurationUtils.cs

+            {
+                return restAuthEntraUsers;
+            }
+            if(!string.IsNullOrWhiteSpace(EntraUserIdentity) && (EntraUserFullInfo != null && EntraUserFullInfo.Length > 0))


don't do such check in this util file

aim-for-better · 2025-06-06T02:55:04Z

src/HDInsight/HDInsight/ClusterConfigurationUtils.cs

+        public static List<EntraUserInfo> GetHDInsightGatewayEntraUser(string EntraUserIdentity, Hashtable[] EntraUserFullInfo)
+        {
+            List<EntraUserInfo> restAuthEntraUsers = new List<EntraUserInfo>();
+            if (String.IsNullOrEmpty(EntraUserIdentity) && (EntraUserFullInfo == null || EntraUserFullInfo.Length == 0))


why allowing both empty or null here and return empty list?

aim-for-better · 2025-06-06T02:58:09Z

src/HDInsight/HDInsight/ManagementCommands/NewAzureHDInsightClusterCommand.cs

+        public string EntraUserIdentity { get; set; }
+
+        [Parameter(
+            HelpMessage = "Gets or sets a list of Entra users as an array of hashtables. Each hashtable should contain keys such as ObjectId, UPN, and DisplayName.")]


here it is UPN but in the Util file you tryGet("Upn")

aim-for-better · 2025-06-06T03:17:42Z

src/HDInsight/HDInsight/help/Add-AzHDInsightClusterIdentity.md

@@ -17,15 +17,15 @@ Adds a cluster identity to a cluster configuration object.
 ```
 Add-AzHDInsightClusterIdentity [-Config] <AzureHDInsightConfig> [-ObjectId] <Guid>
 [-CertificateFilePath] <String> [-CertificatePassword] <String> [[-AadTenantId] <Guid>]
- [[-ApplicationId] <Guid>] [-DefaultProfile <IAzureContextContainer>]
+ [[-ApplicationId] <Guid>] [-DefaultProfile <IAzureContextContainer>] [-ProgressAction <ActionPreference>]
 [<CommonParameters>]


did you change this command?

aim-for-better · 2025-06-06T03:18:07Z

src/HDInsight/HDInsight/help/Add-AzHDInsightClusterIdentity.md

 [<CommonParameters>]
 ```

 ### CertificateFileContents
 ```
 Add-AzHDInsightClusterIdentity [-Config] <AzureHDInsightConfig> [-ObjectId] <Guid>
 [-CertificateFileContents] <Byte[]> [-CertificatePassword] <String> [[-AadTenantId] <Guid>]
- [[-ApplicationId] <Guid>] [-DefaultProfile <IAzureContextContainer>]
+ [[-ApplicationId] <Guid>] [-DefaultProfile <IAzureContextContainer>] [-ProgressAction <ActionPreference>]


did you change this?

aim-for-better · 2025-06-06T03:18:40Z

src/HDInsight/HDInsight/help/Add-AzHDInsightClusterIdentity.md

@@ -208,6 +208,21 @@ Accept pipeline input: True (ByValue)
 Accept wildcard characters: False
 ```

+### -ProgressAction


don't need to change this command right?

aim-for-better · 2025-06-06T03:19:06Z

src/HDInsight/HDInsight/help/Add-AzHDInsightComponentVersion.md

@@ -15,7 +15,7 @@ Adds a version for a service running in a cluster to a cluster configuration obj

 ```
 Add-AzHDInsightComponentVersion [-Config] <AzureHDInsightConfig> [-ComponentName] <String>
- [-ComponentVersion] <String> [-DefaultProfile <IAzureContextContainer>]
+ [-ComponentVersion] <String> [-DefaultProfile <IAzureContextContainer>] [-ProgressAction <ActionPreference>]


check all the commands which we shall not change

src/HDInsight/HDInsight/help/New-AzHDInsightCluster.md

aim-for-better · 2025-06-06T03:21:16Z

src/HDInsight/HDInsight/help/New-AzHDInsightCluster.md

-Accept pipeline input: False
-Accept wildcard characters: False
-```
-
 ### -Version
 Specifies the HDI version of the HDInsight cluster.


Add more examples for this command

v-yukunli and others added 9 commits May 16, 2025 16:57

add cmd: Set-AzHDInsightGatewaySettings ; Get-AzHDInsightClusterGatew…

cbbf936

…ayEntraUserInfo ; New-AzHDInsightClusterGatewayEntraUserInfo modify cmd: New-AzHDInsightCluster

update HDInsight.csproj

eb3b5ce

update HDInsight.csproj

62abc3a

update test

5dbb7c3

update test

59fb028

update test

9c3da94

update test

50748d3

update sessionRecords

305ee7e

Merge branch 'Azure:main' into hdi-ps-sdk-2025-01-15

ae54404

Copilot AI review requested due to automatic review settings May 26, 2025 06:28

2258093686 marked this pull request as draft May 26, 2025 06:28

Copilot AI reviewed May 26, 2025

View reviewed changes

src/HDInsight/HDInsight/ClusterCreateHelper.cs Outdated Show resolved Hide resolved

v-yukunli added 7 commits May 28, 2025 14:09

change command

24646c9

change NewAzureCmd ,SetGatewayCmd

dab854c

run test

7ea2ce3

update entra user parameter

72fab04

playback test

3032d57

update test

257a2bd

update help

abda54b

aim-for-better reviewed Jun 6, 2025

View reviewed changes

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightClusterTests.cs Outdated

Comment on lines 41 to 42

Copy link

Member

aim-for-better Jun 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change to 1 line?

aim-for-better reviewed Jun 6, 2025

View reviewed changes

src/HDInsight/HDInsight.Test/ScenarioTests/HDInsightClusterTests.ps1 Outdated

Comment on lines 52 to 55

Copy link

Member

aim-for-better Jun 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reduce the empty line

aim-for-better reviewed Jun 6, 2025

View reviewed changes

src/HDInsight/HDInsight/help/New-AzHDInsightCluster.md Show resolved Hide resolved

aim-for-better reviewed Jun 6, 2025

View reviewed changes

v-yukunli added 2 commits June 6, 2025 19:57

update test , help

8fe4226

update test

ef9bffa

[HDInsight] Update api version to 2025-01-15-preview #27837

Are you sure you want to change the base?

[HDInsight] Update api version to 2025-01-15-preview #27837

Uh oh!

Conversation

2258093686 commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Mandatory Checklist

Uh oh!

azure-client-tools-bot-prd bot commented May 26, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

2258093686 commented May 26, 2025 •

edited

Loading