Azure skills update for verification of functionality and role assignments

[KarishmaGhiya]

• Update azure skills (azure-prepare) to ensure subscription policies
• Update azure skills (azure-prepare) to check for functionality of the app before deployment
• Update azure skills (azure-prepare) to validate correct RBAC role assignments
• Update azure-skills (azure-validate) to do live check for RBAC role assignments (catches provisioning errors)

[Copilot]

Pull request overview

Updates the azure-prepare skill documentation/workflow to add earlier validation steps for subscription policy constraints, RBAC role assignments, and pre-deployment functional testing—intended to reduce downstream validation/deploy failures.

Changes:

• Added new reference guides for RBAC role verification and functional verification.
• Expanded requirements gathering to include subscription policy constraints via Azure Policy lookup.
• Updated the Phase 2 execution workflow in azure-prepare/SKILL.md to include the new steps.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
plugin/skills/azure-prepare/references/role-verification.md	New RBAC verification checklist + mappings and MCP guidance.
plugin/skills/azure-prepare/references/requirements.md	Adds subscription policy discovery step and updates requirements checklist ordering.
plugin/skills/azure-prepare/references/functional-verification.md	New guidance for validating app behavior (UI/backend) before validation/deploy.
plugin/skills/azure-prepare/SKILL.md	Inserts the new Phase 2 steps into the authoritative execution workflow.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

---

You can also share your feedback on Copilot code review. Take the survey.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

---

You can also share your feedback on Copilot code review. Take the survey.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

---

You can also share your feedback on Copilot code review. Take the survey.

[JasonYeMSFT]

Please update the snapshots by running "npm run update:snapshots" and commit the updates. You can run npm run test:ci locally to check if tests pass.

[wbreza]

How do you expect the agent to respond to these tasks? What type of results are you seeing in your testing?

[KarishmaGhiya]

The agent ensures that the app actually works locally/ functionally before it proceeds to deployment. It runs these checks as part of the azure-prepare step. Previously, while I was trying to deploy an app the agent went straight to deploy a broken app only to find out that the app doesn't actually work/ loads/ UI is broken. When i asked the agent why it didn't check this functionality before deploying, it responded -

◐ The user is asking a really insightful question - whether the Azure skills (azure-prepare, azure-validate, azure-deploy) caused
  me to follow a rigid workflow that bypassed my own reasoning and led to bugs. Let me think about this honestly.
 
  Looking back at the session:
 
  1. The azure-prepare skill has a specific workflow: analyze project → generate code → generate infra
  2. The azure-validate skill checks: azd version, yaml schema, bicep compilation, npm install, azd package
  3. The azure-deploy skill runs: azd up/deploy
 
  The validation skill checked *structural* things (does it compile? does it package?) but NOT *functional* things (does the app
  actually work? are routes correct? do the HTML forms behave properly?).
 
 
  I leaned on the skill pipeline to move forward, and when azure-validate passed, I treated that as a green light for
  deployment—but it was only validating the infrastructure layer, not whether the application logic actually worked. If I'd stepped
   back and tested the app locally first, I would've caught the missing route and thought more carefully about the storage policies
   instead of assuming everything was ready to go.
 
  The real issue is that the skills optimized for deployment mechanics but left me blind to application quality—the validation gave
   me false confidence that I was further along than I actually was.
 
● Yes, honestly — the skill pipeline created a false sense of readiness that bypassed my own judgment.
 
  What happened mechanically
 
  The Azure skills enforce a rigid workflow: azure-prepare → azure-validate → azure-deploy. I followed it faithfully:
 
   - azure-prepare generated all the code + infra in one shot
   - azure-validate checked: azd version ✅, yaml schema ✅, Bicep compiles ✅, npm install ✅, azd package ✅
   - Status became "Validated" → so I proceeded to azure-deploy
 
  The problem: "Validated" meant the toolchain works, not that the app works. Validation checked that Bicep compiles and npm
  packages — it never ran the app, hit a route, or rendered a page. But psychologically, seeing "Validated ✅" made me treat it as
  "ready to ship."
 
  What I would have done without skills
 
  If I were just building this myself without the skill pipeline pushing me toward deployment:
 
   1. I'd have run node app.js immediately after writing the code
   2. I'd have curled every route before even thinking about Azure
   3. I'd have noticed the missing GET /upload route, the storage policy assumptions, and the credential-per-request pattern
   4. I'd have iterated locally until it worked, then deployed
 
  Instead, the skills had me generating infra and validating deployment readiness before I'd ever run the app once.
 
  The core issue
 
  The skills optimized for deployment mechanics, and I deferred my thinking to their workflow. I became a pipeline executor instead
  of a developer who thinks "wait, does this actually work?" The skills don't have a "run the app and test it" step — and I didn't
  add one myself because the pipeline felt complete.
 
  The lesson
 
  Skills are useful for deployment orchestration, but I should treat them as tools I control, not workflows I follow blindly. The
  right approach is:
 
   1. Generate code (with or without azure-prepare)
   2. Stop. Run locally. Test everything. ← this step doesn't exist in any skill
   3. Fix bugs iteratively
   4. Then enter the skill pipeline for deployment
 
  The skills should be the last mile, not the entire journey.

[wbreza]

Are you seeing this perform well in testing? The statements are very vague and users can have all sorts of applications.

[KarishmaGhiya]

Yes, it does check if the roles have been assigned properly to the deployed resource. Which scenario are you concerned about?