[Skill Bug]: Deployment scenario taking too many attempts

[charris-msft]

Preflight Checklist

• I have installed the latest version of GitHub Copilot for Azure in VS Code.
• I have checked existing resources, including using the /help command in the chat window.
• I have searched for similar issues.

Repro Steps

1. Clone this repo: https://github.com/charris-msft/zava_photo_uploader
2. Open Copilot CLI
3. Add the Azure Skills Plugin to Copilot CLI
4. run /mcp reload to load the mcp servers
5. submit the prompt

Prompt

Help me deploy this application to Azure

Expected Result

The simple application deploys smoothly in without any need to troubleshoot

Actual Result

Opus 4.6

Storage account access key policy error - didn't call policy tool, but did see those instructions in best practices when asked.

Sonnet 4.6

Here are the 4 fix loops in order:

Fix 1 — azure.yaml missing language field
ERROR: must specify language or image
The generated azure.yaml had host: containerapp but no language field. AZD 1.23.5 requires it even when you're using Docker. Added language: python.

Fix 2 — Interactive prompt blocked --no-prompt
azd provision --preview --no-prompt hung waiting for input. Because infra/main.bicep uses targetScope = 'resourceGroup', AZD needed a resource group name and dropped into an interactive prompt — which --no-prompt can't satisfy. Fixed by setting $env:AZURE_RESOURCE_GROUP = 'rg-zava-photo-uploader'.

Fix 3 — Azure policy denied shared key access on Storage
ERROR: Policy disallows resource: allowSharedKeyAccess must be false
The subscription has a "Safe Secrets Standard" deny policy requiring allowSharedKeyAccess: false on all Storage accounts. The generated Bicep omitted this property (it defaults to true). Added allowSharedKeyAccess: false to the storage resource.

Fix 4 — Wrong Docker context path in azure.yaml
azd package failed because Docker couldn't find the context. The original config had:
project: ./src
docker:
path: ./src/Dockerfile
context: ./src
AZD resolves docker.path and docker.context relative to project:, so the effective path became ./src/./src — doesn't exist. Fixed to:
docker:
path: Dockerfile
context: .

---

Fixes 1, 2, and 4 are generation mistakes — wrong defaults or undocumented AZD behavior. Fix 3 is an environment awareness gap — the subscription policy was discoverable before writing any Bicep if the model had used the Policy tool in the Azure MCP but wasn't checked.

Attachments

No response

Regression?

No response

Additional Context

No response

[github-actions]

Thanks for the detailed bug report, @charris-msft — the step-by-step fix loop breakdown is really helpful for understanding where things went wrong! 🙏

This has been labeled as a bug under the skills and deploy categories since it describes incorrect code generation during an Azure deployment scenario. The context-awareness label was also added because Fix 3 highlights an environment awareness gap — the skill should have proactively checked subscription policies (via the Azure Policy MCP tool) before generating Bicep, rather than discovering the constraint through a deployment failure.

The team will review the generation mistakes (azure.yaml language field, Docker context path) and the policy pre-check gap to improve the deployment skill's reliability.

Generated by Issue Triage for issue #1222

[tmeschter]

@charris-msft I just ran through this scenario and the only problem I ran into was the fourth, "Wrong Docker context path in azure.yaml". We do have an instruction in azure-validate to call azd package --dry-run which would have caught this problem, but the LLM skipped that step. This is something I've seen a few times now--the LLM will read the azure-validate skill but won't follow all the steps. I'm going to try updating the skill so it adds explicit TODO items for each validation check to the .azure/plan.md file. That is being tracked as the fix for #1421.