gcchigh get-dlp compliancePolicy and Error running Get-PrivilegedUser:

[homebrewjc]

Prerequisites

• This issue has an informative and human-readable title.

ScubaGear Version

Loop through each tenant and run ScubaGear

foreach ($tenant in $tenants) {
Write-Host "Running ScubaGear for $($tenant.Organization)..." -ForegroundColor Cyan

# Run SCuBA with certificate authentication
Invoke-SCuBA `
    -ProductNames @("defender", "teams", "exo", "sharepoint", "aad") `
    -M365Environment $tenant.Environment `
    -AppID $tenant.AppID `
    -CertificateThumbprint $thumbprint `
    -Organization $tenant.Organization `
    -OutPath $tenant.OutputPath `
    -Quiet

# Get the most recent results folder
$mostRecentFolder = Get-ChildItem -Path $tenant.OutputPath -Directory -Filter "M365BaselineConformance_*" | 
                    Sort-Object CreationTime -Descending | 
                    Select-Object -First 1

if ($mostRecentFolder) {
    # Convert JSON files to OEM encoding
    Write-Host "Converting JSON files to OEM encoding for $($tenant.Organization)..." -ForegroundColor Cyan
    Convert-JsonFilesToOEM -FolderPath $mostRecentFolder.FullName
}

# Clean up old report folders after running SCuBA
Write-Host "Cleaning up old report folders for $($tenant.Organization)..." -ForegroundColor Cyan
Clean-OldReports -FolderPath $tenant.OutputPath
    
Write-Host "Completed scan for $($tenant.Organization)" -ForegroundColor Green
Write-Host "----------------------------------------" -ForegroundColor Yellow

}

Write-Host "All tenant scans completed!" -ForegroundColor Green

Operating System

Windows Server 2019 windows 10

PowerShell Version

7.0+

M365 Environment and License(s)

We are using comm and Gcchigh

🐛 Summary

we are running scuba perfectly against our commercial account but get a dlp issue for all gov accounts and get this error for only 2 of our gov accounts. Get-PrivilegedUser: Response status code does not indicate success: Forbidden (Forbidden).

Steps to reproduce

we set this up with a thumbprint and used this script to invoke scuba on each and every account

Function to clean up old report folders

function Clean-OldReports {
param (
[string]$FolderPath
)

# Get all M365BaselineConformance folders
$reportFolders = Get-ChildItem -Path $FolderPath -Directory -Filter "M365BaselineConformance_*"

# If there are more than 1 folders
if ($reportFolders.Count -gt 1) {
    # Sort by creation time, newest first
    $sortedFolders = $reportFolders | Sort-Object CreationTime -Descending
    
    # Keep the most recent folder, remove others
    for ($i = 1; $i -lt $sortedFolders.Count; $i++) {
        Write-Host "  Removing old report folder: $($sortedFolders[$i].FullName)" -ForegroundColor Yellow
        Remove-Item -Path $sortedFolders[$i].FullName -Recurse -Force
    }
    
    Write-Host "  Kept most recent report folder: $($sortedFolders[0].Name)" -ForegroundColor Green
}

}

Function to convert JSON files to OEM encoding

function Convert-JsonFilesToOEM {
param (
[string]$FolderPath
)

# Get all JSON files in the folder and its subfolders
$jsonFiles = Get-ChildItem -Path $FolderPath -Filter "*.json" -Recurse

foreach ($file in $jsonFiles) {
    Write-Host "  Converting $($file.Name) to OEM encoding" -ForegroundColor Cyan
    
    # Read the content with the current encoding
    $content = Get-Content -Path $file.FullName -Raw
    
    # Write the content back with OEM encoding
    $content | Out-File -FilePath $file.FullName -Encoding OEM -Force
}

Write-Host "  Converted $($jsonFiles.Count) JSON files to OEM encoding" -ForegroundColor Green

}

Loop through each tenant and run ScubaGear

foreach ($tenant in $tenants) {
Write-Host "Running ScubaGear for $($tenant.Organization)..." -ForegroundColor Cyan

# Run SCuBA with certificate authentication
Invoke-SCuBA `
    -ProductNames @("defender", "teams", "exo", "sharepoint", "aad") `
    -M365Environment $tenant.Environment `
    -AppID $tenant.AppID `
    -CertificateThumbprint $thumbprint `
    -Organization $tenant.Organization `
    -OutPath $tenant.OutputPath `
    -Quiet

# Get the most recent results folder
$mostRecentFolder = Get-ChildItem -Path $tenant.OutputPath -Directory -Filter "M365BaselineConformance_*" | 
                    Sort-Object CreationTime -Descending | 
                    Select-Object -First 1

if ($mostRecentFolder) {
    # Convert JSON files to OEM encoding
    Write-Host "Converting JSON files to OEM encoding for $($tenant.Organization)..." -ForegroundColor Cyan
    Convert-JsonFilesToOEM -FolderPath $mostRecentFolder.FullName
}

# Clean up old report folders after running SCuBA
Write-Host "Cleaning up old report folders for $($tenant.Organization)..." -ForegroundColor Cyan
Clean-OldReports -FolderPath $tenant.OutputPath
    
Write-Host "Completed scan for $($tenant.Organization)" -ForegroundColor Green
Write-Host "----------------------------------------" -ForegroundColor Yellow

}

Write-Host "All tenant scans completed!" -ForegroundColor Green

Expected behavior

We expected the normal output with out omitting commands

Output from Initialize-SCuBA (optional)

WARNING: Invalid Teams Environment name for Migration API
WARNING: Error running Get-PrivilegedUser: Response status code does not indicate success: Forbidden (Forbidden).
at Invoke-GraphDirectly,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Providers\ExportAADProvid
er.psm1: line 54
at Get-PrivilegedUser,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Providers\ExportAADProvid
er.psm1: line 336
at TryCommand,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Providers\ProviderHelpers
\CommandTracker.psm1: line 23
at Export-AADProvider,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Providers\ExportAADProvid
er.psm1: line 117
at Invoke-ProviderList,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Orchestrator.psm1: line
617
at Invoke-SCuBA,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Orchestrator.psm1: line
403
at , C:\Users\Administrator\Documents\scubaCommerical.ps1: line 262
at , : line 1
WARNING: Error running Get-PrivilegedRole: Response status code does not indicate success: Forbidden (Forbidden).
at Invoke-GraphDirectly,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Providers\ExportAADProvid
er.psm1: line 54
at Get-PrivilegedRole,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Providers\ExportAADProvid
er.psm1: line 688
at TryCommand,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Providers\ProviderHelpers
\CommandTracker.psm1: line 23
at Export-AADProvider,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Providers\ExportAADProvid
er.psm1: line 152
at Invoke-ProviderList,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Orchestrator.psm1: line
617
at Invoke-SCuBA,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Orchestrator.psm1: line
403
at , C:\Users\Administrator\Documents\scubaCommerical.ps1: line 262
at , : line 1
WARNING: Error running Connect-IPPSSession: UnAuthorized
at Connect-ExchangeOnline,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ExchangeOnlineManagement\3.4.0\netFramework\ExchangeOnlineMa
nagement.psm1: line 766
at Connect-IPPSSession,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ExchangeOnlineManagement\3.4.0\netFramework\ExchangeOnlineMa
nagement.psm1: line 904
at Connect-DefenderHelper,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Connection\ConnectHelpers
.psm1: line 126
at Export-DefenderProvider,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Providers\ExportDefenderP
rovider.psm1: line 93
at Invoke-ProviderList,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Orchestrator.psm1: line
623
at Invoke-SCuBA,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Orchestrator.psm1: line
403
at , C:\Users\Administrator\Documents\scubaCommerical.ps1: line 262
at , : line 1
WARNING: Omitting the following commands: Get-DlpCompliancePolicy, Get-DlpComplianceRule, and Get-ProtectionAlert.

[buidav]

@homebrewjc For GCCHigh tenants specifically, ensure your Service Principal Application has the Exchange.ManageAsApp permission from the
Microsoft Exchange Online Protection API as well.

The resulting API permissions for your App Registration should have the Exchange.ManageAsApp API permission from both the Microsoft Exchange Online Protection and Office 365 Exchange Online APIs.

Image of an example App Registration with the permissions below.

[homebrewjc]

Thanks for getting back to me so quick, i just want to clarify that i only need 13 permissions to run scuba, I see 14 graph permissions in your image, are those im missing needed?

and how do i fix the Get-PrivilegedUser, the error is only happening on 2 of the tenants im running scuba on both gcch, but the rest of the gcch dont have this issue.

WARNING: Invalid Teams Environment name for Migration API
WARNING: Error running Get-PrivilegedUser: Response status code does not indicate success: Forbidden (Forbidden).
at Invoke-GraphDirectly,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Providers\ExportAADProvid
er.psm1: line 54
at Get-PrivilegedUser,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Providers\ExportAADProvid
er.psm1: line 336
at TryCommand,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Providers\ProviderHelpers
\CommandTracker.psm1: line 23
at Export-AADProvider,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Providers\ExportAADProvid
er.psm1: line 117
at Invoke-ProviderList,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Orchestrator.psm1: line
617
at Invoke-SCuBA,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0.1741012484\Modules\Orchestrator.psm1: line
403
at , C:\Users\Administrator\Documents\scubaCommerical.ps1: line 262
at , : line 1
WARNING: Error running Get-PrivilegedRole: Response status code does not indicate success: Forbidden (Forbidden).
at Invoke-GraphDirectly,

[schrolla]

@homebrewjc Glad that the permissions fixed the EXO related issue. To follow up, the 14th permission Applications.Read.All was added in our 1.5.0 release to support adding identification of risky Applications into the Azure AD/ Entra report. I note from your module path that you appear to be using a test version of v1.5.0 rather than the release version (which may not have the risky app reporting). So, yes, the 14th permission is needed for the latest release, and I'd recommend you update ScubaGear to the v1.5.0 full release version to get the additional report information, if desired.

As for the Get-PrivilegedUser and Get-PrivilegedRole, you have the scopes required for those commands: Directory.Read.All and RoleManagement.Read.Directory based on your screenshots above. It's possible that updating to the 1.5.0 release version may resolve the issue, so I'd recommend trying that first to see if it addresses the problem.

[homebrewjc]

Awesome i was able to get all working on except one the azure report for this instance. there are no other errors just this.
Fatal Error involving the Report Creation.
Ending ScubaGear execution. Error: Invalid JSON primitive: .

at New-Report,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0\Modules\CreateReport\CreateReport.psm1:
line 66
at Invoke-ReportCreation,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0\Modules\Orchestrator.psm1: line 1277
at Invoke-SCuBA,
C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0\Modules\Orchestrator.psm1: line 440
at , C:\Users\Administrator\Documents\gcc1test.ps1: line 6
at , : line 1
At C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0\Modules\Orchestrator.psm1:1371 char:13

•         throw $InvokeReportErrorMessage
  

•         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  

  • CategoryInfo : OperationStopped: (Fatal Error inv...o file>: line 1:String) [], RuntimeException
  • FullyQualifiedErrorId : Fatal Error involving the Report Creation.
    Ending ScubaGear execution. Error: Invalid JSON primitive: .

  at New-Report, C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0\Modules\CreateReport\Cre
  ateReport.psm1: line 66
  at Invoke-ReportCreation, C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0\Modu
  les\Orchestrator.psm1: line 1277
  at Invoke-SCuBA, C:\Users\Administrator\Documents\WindowsPowerShell\Modules\ScubaGear\1.5.0\Modules\Orche
  strator.psm1: line 440
  at , C:\Users\Administrator\Documents\gcc1test.ps1: line 6
  at , : line 1

[schrolla]

Awesome i was able to get all working on except one the azure report for this instance. there are no other errors just this. Fatal Error involving the Report Creation. Ending ScubaGear execution. Error: Invalid JSON primitive: .

Glad to hear the tool is working better for you now.

That particular error usually indicates that there was some sort of processing error such that when the configuration settings were extracted from Azure AD/Entra, one of the values was unexpectedly null or empty which resulted in the ScubaResults JSON file being malformed. Which is what the "Invalid JSON primitive" error means.

It has also come up with certain escape sequences or special characters in setting values such as '' or unicode characters. Although we've closed out the known bugs.

Usually, the best way to isolate the issue is to inspect the produced ScubaResults JSON file to find the malformed line and associated JSON key. This points to which setting or value might be causing the issue. Often, it's a line where there is a key defined, but no value. It often looks something like this. Some editors (like VSCode) will also parse and highlight areas that are malformed to make it easier to find the malformed area.

"key": ,

If you could find that key and share it, we might be able to figure out if it is a known issue and if there is a workaround or file a bug report to fix if it is new.

[homebrewjc]

This is the error i found. it looks like "risky_third_party_service_principals": , needs a value like "risky_third_party_service_principals": [],

[schrolla]

This is the error i found. it looks like "risky_third_party_service_principals": , needs a value like "risky_third_party_service_principals": [],

Yes, that definitely looks like the culprit and gives us a better idea of where to investigate the issue. I've created #1658 to track the bug.

For the time being, if you want a full report from that tenant, you can set the value in the JSON to an empty list [] and run ScubaGear again using the Invoke-ScubaCached command which will reprocess the fixed JSON and generate a set of reports from it. This command uses the same parameters as Invoke-SCuBA with a few extra to tell it to use the existing results file rather than reaching out to the tenant, but still does the assess and report stage based on the results file.

The full command would be as follows where <previous parameters> would be the same as the original Invoke-SCuBA command:

Invoke-ScubaCached -ExportProvider:$false -Login:$false <previous parameters>

That should give you a report manually until we can isolate and resolve the bug.

[mitchelbaker-cisa]

@homebrewjc If you'd like to test it out, you can download this feature branch with a fix for the risky service principals issue: https://github.com/cisagov/ScubaGear/archive/refs/heads/1607-risky-sps-invalid-json.zip

Let us know if you have any other questions.