feat: Refactor error handling and update parameter retrieval methods

[Saswato-Microsoft]

Purpose

This pull request introduces several reliability and robustness improvements across deployment scripts and configuration files, with a focus on handling transient errors, improving workspace creation logic, and ensuring proper parameterization. The most significant changes include enhanced retry logic for Azure deployments, improved error handling for workspace creation, and connection timeout retries for OneLakeIndex scripts.

Deployment robustness improvements:

• Added retry logic for the AI Landing Zone deployment in preprovision-integrated.ps1, including polling for account provisioning state, cleaning up failed deployments before retry, and limiting retries to three attempts. This helps handle transient errors such as AccountProvisioningStateInvalid and ensures deployments only fail after multiple attempts. [1] [2] [3]
• Updated temporary directory handling to be more robust across environments, using $env:TEMP, $env:TMPDIR, or /tmp as needed.

Workspace creation error handling:

• Improved workspace creation logic in create_fabric_workspace.ps1 to handle 409 Conflict errors by retrying workspace lookup via /workspaces and /groups endpoints, ensuring existing workspaces are properly identified and reused. [1] [2]
• Fixed a typo in the variable name ($pincipalId → $principalId) when assigning admin roles to workspaces, ensuring correct role assignment. [1] [2]

Connection timeout handling for OneLakeIndex scripts:

• Added retry logic for connection timeouts in OneLakeIndex scripts (02_create_onelake_skillsets.ps1, 03_create_onelake_index.ps1, 04_create_onelake_datasource.ps1, 05_create_onelake_indexer.ps1), allowing the scripts to wait and retry when the search service data plane is not yet available. [1] [2] [3] [4]

Parameterization and configuration fixes:

• Changed principalId in infra/main.bicepparam to be read from the AZURE_PRINCIPAL_ID environment variable, improving flexibility and automation.
• Set a default value for fabricCapacityAdmins in infra/main.bicepparam to avoid empty array issues.
• Corrected the path case for the PostgreSQL mirroring preparation script in azure.yaml to match the actual directory structure.

Does this introduce a breaking change?

• Yes
• No

Golden Path Validation

• I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

• I have validated the deployment process successfully and all services are running as expected with this change.

What to Check

Verify that the following are valid

• ...

Other Information