diff --git a/plugin/skills/azure-deploy/SKILL.md b/plugin/skills/azure-deploy/SKILL.md index ee8b415dd..6b8bf5e12 100644 --- a/plugin/skills/azure-deploy/SKILL.md +++ b/plugin/skills/azure-deploy/SKILL.md @@ -4,7 +4,7 @@ description: "Execute Azure deployments for ALREADY-PREPARED applications that h license: MIT metadata: author: Microsoft - version: "1.0.5" + version: "1.0.7" --- # Azure Deploy @@ -65,6 +65,7 @@ Activate this skill when user wants to: | 5 | **Post-Deploy** — Configure SQL managed identity and apply EF migrations if applicable | [Post-Deployment](references/recipes/azd/post-deployment.md) | | 6 | **Handle Errors** — See recipe's `errors.md` | — | | 7 | **Verify Success** — Confirm deployment completed and endpoints are accessible | [Verification](references/recipes/azd/verify.md) | +| 8 | **Live Role Verification** — Query Azure to confirm provisioned RBAC roles are correct and sufficient | [live-role-verification.md](references/live-role-verification.md) | > **⛔ VALIDATION PROOF CHECK** > @@ -82,6 +83,7 @@ Activate this skill when user wants to: | `mcp_azure_mcp_subscription_list` | List available subscriptions | | `mcp_azure_mcp_group_list` | List resource groups in subscription | | `mcp_azure_mcp_azd` | Execute AZD commands | +| `azure__role` | List role assignments for live RBAC verification (step 8) | ## References diff --git a/plugin/skills/azure-deploy/references/live-role-verification.md b/plugin/skills/azure-deploy/references/live-role-verification.md new file mode 100644 index 000000000..2d0130bc6 --- /dev/null +++ b/plugin/skills/azure-deploy/references/live-role-verification.md @@ -0,0 +1,102 @@ +# Live Role Verification + +Query Azure to confirm that provisioned RBAC role assignments are correct and sufficient for the application to function. This complements the static role check in azure-validate by validating **live Azure state**. + +## How It Differs from azure-validate's Role Check + +| Check | Skill | What It Verifies | +|-------|-------|-----------------| +| **Static** | azure-validate | Generated Bicep/Terraform has correct role assignments in code | +| **Live** | azure-deploy (this) | Provisioned Azure resources actually have the right roles assigned | + +Both checks are needed because: +- Bicep may be correct but provisioning could fail silently for roles +- Manual changes or policy enforcement may alter role assignments +- Previous deployments may have stale or conflicting roles + +## When to Run + +After deployment verification (step 7). Resources are now provisioned, so live role assignments can be queried. + +## Verification Steps + +### 1. Identify App Identities + +Read `.azure/plan.md` to find all services with managed identities. Then query Azure for their principal IDs: + +```bash +# App Service +az webapp identity show --name -g --query principalId -o tsv + +# Container App +az containerapp identity show --name -g --query principalId -o tsv + +# Function App +az functionapp identity show --name -g --query principalId -o tsv +``` + +### 2. Query Live Role Assignments + +Use MCP tools to list role assignments for each resource **and identity** (using the `principalId` from step 1): + +``` +azure__role( + command: "role_assignment_list", + scope: "", + assignee_object_id: "" +) +``` + +Or via CLI: + +```bash +az role assignment list --scope --assignee-object-id --output table +``` + +### 3. Cross-Check Against Requirements + +For each identity, verify the assigned roles match what the app needs: + +| App Operation | Expected Role | Scope | +|---------------|---------------|-------| +| Read/write blobs | Storage Blob Data Contributor | Storage account | +| Generate user delegation SAS | Storage Blob Delegator | Storage account | +| Read secrets | Key Vault Secrets User | Key Vault | +| Send messages | Azure Service Bus Data Sender | Service Bus namespace | +| Read/write documents | Cosmos DB Built-in Data Contributor | Cosmos DB account | + +### 4. Check for Common Issues + +| Issue | How to Detect | Fix | +|-------|---------------|-----| +| Role assigned at wrong scope | Role on resource group but needed on specific resource | Reassign at resource scope | +| Generic role instead of data role | `Contributor` assigned but no data-plane access | Replace with data-plane role (e.g., `Storage Blob Data Contributor`) | +| Missing role entirely | No assignment found for identity on target resource | Add role assignment to Bicep and redeploy | +| Stale role from previous deployment | Old principal ID with roles, new identity without | Clean up old assignments, add new ones | + +## Decision Tree + +``` +Resources provisioned? +├── No → Skip live check (nothing to query yet) +└── Yes → For each app identity: + ├── Query role assignments on target resources + ├── Compare against expected roles from plan + │ ├── All roles present and correct → ✅ Pass + │ ├── Missing roles → ❌ Fail — add to Bicep, redeploy + │ └── Wrong scope or generic roles → ⚠️ Warning — fix and redeploy + └── Record results in deployment verification +``` + +## Record in Deployment Verification + +Add live role verification results to the deployment log in `.azure/plan.md`: + +```markdown +### Live Role Verification +- Command: `az role assignment list --scope --assignee-object-id ` +- Results: + - → on ✅ + - → missing on ❌ +- Status: Pass / Fail +``` diff --git a/plugin/skills/azure-prepare/SKILL.md b/plugin/skills/azure-prepare/SKILL.md index 454d56807..0a6e8ba7b 100644 --- a/plugin/skills/azure-prepare/SKILL.md +++ b/plugin/skills/azure-prepare/SKILL.md @@ -4,7 +4,7 @@ description: "Prepare Azure apps for deployment (infra Bicep/Terraform, azure.ya license: MIT metadata: author: Microsoft - version: "1.0.6" + version: "1.0.7" --- # Azure Prepare @@ -103,8 +103,9 @@ Execute the approved plan. Update `.azure/plan.md` status after each step. | 2 | **Confirm Azure Context** — Detect and confirm subscription + location and check the resource provisioning limit | [Azure Context](references/azure-context.md) | | 3 | **Generate Artifacts** — Create infrastructure and configuration files | [generate.md](references/generate.md) | | 4 | **Harden Security** — Apply security best practices | [security.md](references/security.md) | -| 5 | **⛔ Update Plan (MANDATORY before hand-off)** — Use the `edit` tool to change the Status in `.azure/plan.md` to `Ready for Validation`. You **MUST** complete this edit **BEFORE** invoking azure-validate. Do NOT skip this step. | `.azure/plan.md` | -| 6 | **⚠️ Hand Off** — Invoke **azure-validate** skill. Your preparation work is done. Deployment execution is handled by azure-deploy. **PREREQUISITE:** Step 5 must be completed first — `.azure/plan.md` status must say `Ready for Validation`. | — | +| 5 | **Functional Verification** — Verify the app works (UI + backend), locally if possible | [functional-verification.md](references/functional-verification.md) | +| 6 | **⛔ Update Plan (MANDATORY before hand-off)** — Use the `edit` tool to change the Status in `.azure/plan.md` to `Ready for Validation`. You **MUST** complete this edit **BEFORE** invoking azure-validate. Do NOT skip this step. | `.azure/plan.md` | +| 7 | **⚠️ Hand Off** — Invoke **azure-validate** skill. Your preparation work is done. Deployment execution is handled by azure-deploy. **PREREQUISITE:** Step 6 must be completed first — `.azure/plan.md` status must say `Ready for Validation`. | — | --- diff --git a/plugin/skills/azure-prepare/references/functional-verification.md b/plugin/skills/azure-prepare/references/functional-verification.md new file mode 100644 index 000000000..28f348d2c --- /dev/null +++ b/plugin/skills/azure-prepare/references/functional-verification.md @@ -0,0 +1,76 @@ +# Functional Verification + +Verify that the application works correctly — both UI and backend — before proceeding to validation and deployment. This step prevents deploying broken or incomplete functionality to Azure. + +## When to Verify + +After generating all artifacts (code, infrastructure, configuration) and applying security hardening — but **before** marking the plan as `Ready for Validation`. + +## Verification Checklist + +Use `ask_user` to confirm functional testing with the user: + +``` +"Before we proceed to deploy, would you like to verify the app works as expected? +We can test both the UI and backend to catch issues before they reach Azure." +``` + +### Backend Verification + +| Check | How | +|-------|-----| +| **App starts without errors** | Run the app and confirm no startup crashes or missing dependencies | +| **API endpoints respond** | Test core routes (e.g., `curl` health, list, create endpoints) | +| **Data operations work** | Verify CRUD operations against storage, database, or other services | +| **Authentication flows** | Confirm auth works (tokens, managed identity fallback, login/logout) | +| **Error handling** | Verify error responses are meaningful (not unhandled exceptions) | + +### UI Verification + +| Check | How | +|-------|-----| +| **Page loads** | Open the app in a browser and confirm the UI renders | +| **Interactive elements work** | Test buttons, forms, file inputs, navigation links | +| **Data displays correctly** | Verify lists, images, and dynamic content render from the backend | +| **User workflows complete** | Walk through the core user journey end-to-end (e.g., upload → view → delete) | + +## Decision Tree + +``` +App artifacts generated? +├── Yes → Ask user: "Would you like to verify functionality?" +│ ├── User says yes +│ │ ├── App can run locally? → Run locally, verify backend + UI +│ │ ├── API-only / no UI? → Test endpoints with curl or similar +│ │ └── Static site? → Open in browser, verify rendering +│ │ Then: +│ │ ├── Works → Proceed to Update Plan (step 6) +│ │ └── Issues found → Fix issues, re-test +│ └── User says no / skip → Proceed to Update Plan (step 6) +└── No → Go back to Generate Artifacts (step 3) +``` + +## Running Locally + +For apps that can run locally, help the user start the app based on the detected runtime: + +| Runtime | Command | Notes | +|---------|---------|-------| +| Node.js | `npm install && npm start` | Set `PORT=3000` if not configured | +| Python | `pip install -r requirements.txt && python app.py` | Use virtual environment | +| .NET | `dotnet run` | Check `launchSettings.json` for port | +| Java | `mvn spring-boot:run` or `gradle bootRun` | Check `application.properties` | + +> ⚠️ **Warning:** For apps using Azure services (e.g., Blob Storage, Cosmos DB), local testing requires the user to be authenticated via `az login` with sufficient RBAC roles, or to have local emulators configured (e.g., Azurite for Storage). + +## Record in Plan + +After functional verification, add a note to `.azure/plan.md`: + +```markdown +## Functional Verification +- Status: Verified / Skipped +- Backend: Tested / Not applicable +- UI: Tested / Not applicable +- Notes: +``` diff --git a/plugin/skills/azure-prepare/references/requirements.md b/plugin/skills/azure-prepare/references/requirements.md index 1d280b146..e2e254d4b 100644 --- a/plugin/skills/azure-prepare/references/requirements.md +++ b/plugin/skills/azure-prepare/references/requirements.md @@ -36,6 +36,27 @@ Collect project requirements through conversation before making architecture dec | Industry regulations | Security controls | | Internal policies | Approval workflows | +### 5. Subscription Policies + +After the user confirms a subscription, query Azure Policy assignments to discover enforcement constraints before making architecture decisions. + +``` +mcp_azure_mcp_policy(command: "policy_assignment_list", subscription: "") +``` + +| Policy Constraint | Impact | +|-------------------|--------| +| Blocked resource types or SKUs | Exclude from architecture | +| Required tags | Add to all Bicep/Terraform resources | +| Allowed regions | Restrict location choices | +| Network restrictions (e.g., no public endpoints) | Adjust networking and access patterns | +| Storage policies (e.g., deny shared key access) | Use policy-compliant auth | +| Naming conventions | Apply to resource naming | + +> ⚠️ **Warning:** Skipping this step can cause deployment failures when Azure Policy denies resource creation. Checking policies here prevents wasted work in architecture and generation phases. + +Record discovered policy constraints in `.azure/plan.md` under a **Policy Constraints** section so they feed into architecture decisions. + ## Gather via Conversation Use `ask_user` tool to confirm each of these with the user: @@ -44,7 +65,8 @@ Use `ask_user` tool to confirm each of these with the user: 2. Expected scale 3. Budget constraints 4. Compliance requirements (including data residency preferences) -5. Architecture preferences (if any) +5. Subscription and policy constraints (confirm subscription, then check policies automatically) +6. Architecture preferences (if any) ## Document in Plan diff --git a/plugin/skills/azure-validate/SKILL.md b/plugin/skills/azure-validate/SKILL.md index a955485f0..9efeea139 100644 --- a/plugin/skills/azure-validate/SKILL.md +++ b/plugin/skills/azure-validate/SKILL.md @@ -4,7 +4,7 @@ description: "Pre-deployment validation for Azure readiness. Run deep checks on license: MIT metadata: author: Microsoft - version: "1.0.0" + version: "1.0.2" --- # Azure Validate @@ -43,10 +43,11 @@ metadata: | 1 | **Load Plan** — Read `.azure/plan.md` for recipe and configuration. If missing → run azure-prepare first | `.azure/plan.md` | | 2 | **Run Validation** — Execute recipe-specific validation commands | [recipes/README.md](references/recipes/README.md) | | 3 | **Build Verification** — Build the project and fix any errors before proceeding | See recipe | -| 4 | **Record Proof** — Populate **Section 7: Validation Proof** with commands run and results | `.azure/plan.md` | -| 5 | **Resolve Errors** — Fix failures before proceeding | See recipe's `errors.md` | -| 6 | **Update Status** — Only after ALL checks pass, set status to `Validated` | `.azure/plan.md` | -| 7 | **Deploy** — Invoke **azure-deploy** skill | — | +| 4 | **Static Role Verification** — Review Bicep/Terraform for correct RBAC role assignments in code | [role-verification.md](references/role-verification.md) | +| 5 | **Record Proof** — Populate **Section 7: Validation Proof** with commands run and results | `.azure/plan.md` | +| 6 | **Resolve Errors** — Fix failures before proceeding | See recipe's `errors.md` | +| 7 | **Update Status** — Only after ALL checks pass, set status to `Validated` | `.azure/plan.md` | +| 8 | **Deploy** — Invoke **azure-deploy** skill | — | > **⛔ VALIDATION AUTHORITY** > diff --git a/plugin/skills/azure-validate/references/role-verification.md b/plugin/skills/azure-validate/references/role-verification.md new file mode 100644 index 000000000..f4291fdec --- /dev/null +++ b/plugin/skills/azure-validate/references/role-verification.md @@ -0,0 +1,75 @@ +# Role Assignment Verification + +Verify that all RBAC role assignments in the generated infrastructure are correct and sufficient before deployment. Incorrect or missing roles are a common cause of runtime failures. + +## When to Verify + +After build verification (step 3) and **before** recording proof (step 5). Role issues surface as cryptic auth errors during deployment — catching them here saves debugging time. + +## Verification Checklist + +Review every resource-to-identity relationship in the generated Bicep/Terraform: + +| Check | How | +|-------|-----| +| **Every service identity has roles** | Each app with a managed identity must have at least one role assignment | +| **Roles match data operations** | Use service-specific **data-plane** roles for data access (see mapping table below); use generic Reader/Contributor/Owner only for management-plane operations | +| **Scope is least privilege** | Roles scoped to specific resources, not resource groups or subscriptions | +| **No missing roles** | Cross-check app code operations against assigned roles (see table below) | +| **Local dev identity has roles** | If testing locally, the user's identity needs equivalent roles via `az login` | + +## Common Service-to-Role Mapping + +| Service Operation | Required Role | Common Mistake | +|-------------------|---------------|----------------| +| Read blobs | Storage Blob Data Reader | Using generic Reader (no data access) | +| Read + write blobs | Storage Blob Data Contributor | Missing write permission | +| Generate SAS via user delegation | Storage Blob Delegator + Data Reader/Contributor | Forgetting Delegator role | +| Read Key Vault secrets | Key Vault Secrets User | Using Key Vault Reader (no secret access) | +| Read + write Cosmos DB | Cosmos DB Built-in Data Contributor | Using generic Contributor | +| Send Service Bus messages | Azure Service Bus Data Sender | Using generic Contributor | +| Read queues | Storage Queue Data Reader | Using Blob role for queues | + +## How to Verify (Static Code Review) + +Review the generated Bicep/Terraform files directly — do **not** query live Azure state here. For each role assignment resource in your infrastructure code: + +1. Identify the **principal** (which managed identity) +2. Identify the **role** (which role definition) +3. Identify the **scope** (which target resource) +4. Cross-check against the app code to confirm the role grants the required data-plane access + +> 💡 **Tip:** Search your Bicep for `Microsoft.Authorization/roleAssignments` or your Terraform for `azurerm_role_assignment` to find all role assignments. + +> ⚠️ **Live role verification** (querying Azure for actually provisioned roles) is handled by **azure-deploy** step 8 as a post-deployment check. This step is a static code review only. + +## Decision Tree + +``` +For each app identity in the generated infrastructure: +├── Has role assignments? +│ ├── No → Add required role assignments to Bicep/Terraform +│ └── Yes → Check each role: +│ ├── Role matches code operations? → ✅ OK +│ ├── Role too broad? → Narrow to least privilege +│ └── Role insufficient? → Upgrade or add missing role +│ +For local testing: +├── User identity has equivalent roles? +│ ├── No → Grant roles via CLI or inform user +│ └── Yes → ✅ Ready for functional verification +``` + +> ⚠️ **Warning:** Generic roles like `Contributor` or `Reader` do **not** include data-plane access. For example, `Contributor` on a Storage Account cannot read blobs — you need `Storage Blob Data Contributor`. This is the most common RBAC mistake. + +## Record in Plan + +After role verification, update `.azure/plan.md`: + +```markdown +## Role Assignment Verification +- Status: Verified / Issues Found +- Identities checked: +- Roles confirmed: +- Issues: +``` diff --git a/tests/azure-deploy/__snapshots__/triggers.test.ts.snap b/tests/azure-deploy/__snapshots__/triggers.test.ts.snap index a55b16632..121767717 100644 --- a/tests/azure-deploy/__snapshots__/triggers.test.ts.snap +++ b/tests/azure-deploy/__snapshots__/triggers.test.ts.snap @@ -38,6 +38,7 @@ exports[`azure-deploy - Trigger Tests Trigger Keywords Snapshot skill descriptio "production", "publish", "push", + "rbac", "recovery", "requires", "runs", @@ -96,6 +97,7 @@ exports[`azure-deploy - Trigger Tests Trigger Keywords Snapshot skill keywords m "production", "publish", "push", + "rbac", "recovery", "requires", "runs", diff --git a/tests/azure-validate/__snapshots__/triggers.test.ts.snap b/tests/azure-validate/__snapshots__/triggers.test.ts.snap index bb93b0ecb..3e0db41e6 100644 --- a/tests/azure-validate/__snapshots__/triggers.test.ts.snap +++ b/tests/azure-validate/__snapshots__/triggers.test.ts.snap @@ -22,6 +22,7 @@ exports[`azure-validate - Trigger Tests Trigger Keywords Snapshot skill descript "pre-deployment", "preflight", "prerequisites", + "rbac", "readiness", "ready", "serverless", @@ -58,6 +59,7 @@ exports[`azure-validate - Trigger Tests Trigger Keywords Snapshot skill keywords "pre-deployment", "preflight", "prerequisites", + "rbac", "readiness", "ready", "serverless",