ci: Enhance Azure Dev workflow with submodule support and service principal configuration

[Harmanpreet-Microsoft]

Purpose

This pull request updates the GitHub Actions workflows for Azure deployments to improve CI/CD automation and reliability, especially for service principal-based deployments. The main changes include ensuring submodules are checked out, dynamically configuring Bicep parameters for service principals, automating resource group and environment setup, and adding a robust retry mechanism for infrastructure provisioning.

Key changes include:

Service Principal and Environment Configuration:

• Added steps in both .github/workflows/azd-template-validation.yml and .github/workflows/azure-dev.yml to dynamically set principalType and principalId in infra/main.bicepparam using environment variables, enabling support for service principal authentication in CI/CD scenarios. [1] [2]
• Added logic to fetch the service principal object ID and set it as an environment variable for use in subsequent steps.
• Automated the creation of the resource group and Azure Developer CLI environment if they do not already exist, improving idempotency and reducing manual setup.

Workflow Reliability and Robustness:

• Implemented a retry mechanism for the infrastructure provisioning step: if the first attempt fails, the workflow re-authenticates, checks the resource group status, waits for deletion if necessary, recreates the group, and retries provisioning.

General Workflow Improvements:

• Ensured all submodules are checked out recursively in both workflows to avoid missing dependencies. [1] [2]
• Set additional environment variables (TEMP, principalId, principalType, etc.) to ensure consistency and compatibility across workflow steps. [1] [2]

These changes make the workflows more robust, automate key setup steps, and better support service principal-based deployments in CI/CD pipelines.

• ...

Does this introduce a breaking change?

• Yes
• No

Golden Path Validation

• I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

• I have validated the deployment process successfully and all services are running as expected with this change.

What to Check

Verify that the following are valid

• ...

Other Information

[Prajwal-Microsoft]

We can change these directly in main.bicepparm and make them configurable instead of changing for a pipeline run alone, I think Saswato might have already done the change, connect with him and check