Eval-driven content fixes: #31, #23, #24, #29 (vally 0.8.0) - #36
Open
David Pine (IEvangelist) wants to merge 7 commits into
Open
Eval-driven content fixes: #31, #23, #24, #29 (vally 0.8.0)#36David Pine (IEvangelist) wants to merge 7 commits into
David Pine (IEvangelist) wants to merge 7 commits into
Conversation
David Pine (IEvangelist)
force-pushed
the
dapine/skill-evals-content
branch
from
July 13, 2026 13:45
a8ed6d8 to
6be4a93
Compare
David Pine (IEvangelist)
force-pushed
the
dapine/skill-evals-content
branch
2 times, most recently
from
July 16, 2026 18:54
235d5a2 to
46a0293
Compare
David Pine (IEvangelist)
force-pushed
the
dapine/skill-evals-content
branch
3 times, most recently
from
July 29, 2026 17:53
ba6e748 to
ef6c30c
Compare
TypeScript `add*`/`with*` builder methods return a fluent `*ResourcePromise` that is both a `PromiseLike` and a chainable builder. Only `createBuilder()` and `build().run()` actually need `await`; intermediate calls should be assigned to a `const` and chained directly. The examples over-used `await`, teaching an awkward orchestration-flavored style. - typescript-authoring.md: de-await skeleton + 10 example snippets; add a new "Promise-fluent return types — when you actually need `await`" section and a Hard Rules anti-pattern row. - apphost-wiring.md: de-await WithReference, cross-service env wiring, container lifetime, and data-volume TS snippets. - javascript-apps.md: de-await dev-script and workspace-monorepo snippets. - SKILL.md: de-await the TS column of the C#-vs-TS quick reference. - evals/eval.yaml: add capability stimulus `aspireify-fluent-ts-001` (tag issue=31) codifying the fluent no-over-await style via output-not-contains (`await builder.add`, `(await builder`) plus a prompt grader; verified green locally (5/5 graders, 2 skills loaded). Fixes #31 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Brownfield JS/TS monorepos commonly have a source-linked shared package (`packages/shared` compiled to `dist/`) consumed by sibling services/apps via npm/yarn/pnpm workspaces. The naive wiring "works" until `tsc --incremental` silently skips emit, leaving `dist/` empty — consumers then fail at first request with a runtime error that looks like an Aspire bug. - javascript-apps.md: new "Shared library packages in monorepos" section — model the build as a one-shot `addJavaScriptApp` resource that consumers `.waitForCompletion()` on; explain `waitForCompletion` vs `waitFor`; a prominent up-front warning plus a dedicated "tsc incremental emit pitfall" subsection with the `prebuild` clean-step fix (and `tsc --build --clean` alternative); and a "when NOT to use this pattern" note for registry-published packages. - SKILL.md: add a Scan-table heuristic row (root `workspaces` + a member whose `main`/`exports` points at `dist/` depended on by another member) and a Propose callout, both pointing at the new section. - evals/eval.yaml: add capability stimulus `aspireify-monorepo-shared-001` (tag issue=23) requiring the shared-build + waitForCompletion wiring AND the tsc-incremental pitfall warning; verified green locally (2/2 runs, 4/4 graders). Fixes #23 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
`aspire agent init` on the 13.4 CLI only shipped the 13.3 breaking-changes scrub list, so agents had no 13.4 reference to scrub generated code, scripts, CI, or deployment snippets against. - references/aspire-13-4-breaking-changes.md: new agent-facing scrub list sourced from the 13.4 release notes — `aspire exec` removal, `aspire ps` flag removals, TypeScript `.aspire/modules/` consolidation, persistent executable/project lifetimes, Kubernetes `WithRoute`→`WithPath` + path-type split + external-endpoint requirement, `WithHelm` consolidation, Azure Front Door naming, Foundry `PublishAsHostedAgent`→`WithComputeEnvironment` and the `AddPromptAgent` parameter reorder, `PublishAsNpmPackageScript`→ `PublishAsPackageScript`, Keycloak HTTPS, named resource-command options, pre-startup TS validation, `Aspire.Hosting.Testing` HTTPS default, RabbitMQ 4.3, `AddNatsClient`, the **PostgreSQL 17→18 data-volume incompatibility**, and a 13.3→13.4 migration checklist. - references/aspire-breaking-changes.md: new version-aware index that points at the right per-version scrub list with upstream release-notes fallbacks. - SKILL.md: wire both new references into the References section. - evals/eval.yaml: add capability stimulus `router-upgrade-13-4-001` (tag issue=24) requiring specific 13.4 breaking changes, the Postgres 18 data-volume warning, and the `aspire update --self` → `aspire update` flow; verified green locally (2/2 runs, 3/3 graders). Fixes #24 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a reference for converting exported Azure infrastructure-as-code (hand-written or `az group export` Bicep / ARM) into an Aspire AppHost: - New references/bicep-to-apphost.md: conversion workflow, an authoritative Bicep `type` → `AddAzure*` integration map, `ConfigureInfrastructure` tweaks, the custom `AddBicepTemplate` fallback for resources Aspire does not model natively, threading Bicep `outputs` as references, "what not to port", a decision tree, and validation via `aspire publish` diffing. - Lead the reference with an IMPORTANT callout covering the two steps agents most often miss: map every natively-supported resource to its `AddAzure*` API, and always raise provision-new vs reference-existing for production infrastructure (`AsExisting` / `PublishAsExisting` / `AddConnectionString`). - Wire the reference into SKILL.md: a Scan-table row for Azure IaC, an imperative "you MUST open bicep-to-apphost.md before proposing" callout, a Propose clarifying question, and References-list entries. - Add capability stimulus `aspireify-bicep-convert-001` plus an `evals/azure-bicep/infra/main.bicep` fixture (storage + Service Bus + Azure Maps) so the agent converts real IaC rather than answering from memory. Verified green: 3/3 trials, all five graders pass. Fixes #29 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rect Issue #31 reduced `await` noise in the TypeScript AppHost examples, but the reference didn't explain *why* passing an un-awaited resource variable to a fluent API (e.g. `withReference(db)`, `waitFor(api)`) is type-correct — only that it works at runtime. That left room to second-guess whether assigned variables still need `await` for their types to line up. Spell out the type-system reason, grounded in the generated SDK: every resource-accepting parameter is generated as `Awaitable<T> = T | PromiseLike<T>` and each `*ResourcePromise` is a `PromiseLike<TheResource>`, so a plain `const db = builder.addPostgres('pg')` is the intended, fully-typed form. Also clarify that the two remaining awaits (`createBuilder()` and `builder.build().run()`) are about execution, not types. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The stimulus asked to wire ServiceDefaults into 'each of my projects' but its environment shipped only MyApp.AppHost/Program.cs. With no service Program.cs to edit, the agent hunted for the Api/Worker projects and wandered past the 600s trial timeout (~1/3 trials), tripping evalHadExecutionErrors and hard-failing the whole eval. - Add minimal MyApp.ApiService/Program.cs and MyApp.Web/Program.cs fixtures (the 'before' state, no ServiceDefaults wired) plus MyApp.Web.csproj so the canonical csharp-apphost fixture matches what the AppHost already references (apiservice + webfrontend). - Mount the two service Program.cs files and the ServiceDefaults .csproj into the stimulus so the agent has concrete files to edit instead of hunting. - Align the adds_service_defaults grader to the AppHost's real projects (ApiService and Web, not a phantom Worker). - Add 'scoring: binary' to the four yes/no prompt graders; under the default scale_1_5 an adequate answer normalised to 0.5, below the 0.7 threshold. Mirrors the router/orchestration hardening in 5ca3eaf. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 788994fa-ffbf-4722-a7b8-0dcc3b174dbf
David Pine (IEvangelist)
force-pushed
the
dapine/skill-evals-content
branch
from
July 30, 2026 11:45
ef6c30c to
a637687
Compare
David Pine (IEvangelist)
changed the base branch from
dapine/skill-evals-vally
to
main
July 30, 2026 11:45
David Pine (IEvangelist)
marked this pull request as ready for review
July 30, 2026 11:45
David Pine (IEvangelist)
requested a review
from Shayne Boyer (spboyer)
as a code owner
July 30, 2026 11:45
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the aspireify and aspire skill content to address eval-driven documentation gaps and correctness issues introduced/validated via the vally 0.8.0 evaluation loop, adding new references and new stimuli/fixtures to gate the intended behaviors.
Changes:
- Updates TypeScript AppHost guidance to prefer fluent
*ResourcePromisechaining (minimizing unnecessaryawait) and adds new eval stimuli to enforce the pattern. - Adds JavaScript/TypeScript monorepo guidance for source-linked shared packages (one-shot build resource +
waitForCompletion, withtsc --incrementalcaveat) and corresponding eval stimulus. - Adds new references for Bicep/ARM IaC → AppHost conversion and Aspire 13.4 breaking-change scrubs, plus eval/fixture updates to validate routing and upgrade guidance.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| skills/aspireify/SKILL.md | Adds Bicep conversion reference + scan/propose heuristics; updates TS quick reference to de-await addProject. |
| skills/aspireify/references/typescript-authoring.md | Rewrites TS examples to avoid over-await; adds an explanation section about fluent promise types. |
| skills/aspireify/references/javascript-apps.md | De-awaits examples; adds “Shared library packages in monorepos” guidance + pitfall callouts. |
| skills/aspireify/references/bicep-to-apphost.md | New reference for mapping Bicep/ARM resources to native Aspire APIs with custom-template fallbacks. |
| skills/aspireify/references/apphost-wiring.md | Updates TS wiring snippets to remove unnecessary await. |
| skills/aspireify/evals/eval.yaml | Adds/updates stimuli and binary-scored graders for fluent TS style, monorepo shared builds, and IaC conversion. |
| skills/aspire/SKILL.md | Adds references to the breaking-changes index + new 13.4 scrub list. |
| skills/aspire/references/aspire-breaking-changes.md | New version-index reference for Aspire breaking changes. |
| skills/aspire/references/aspire-13-4-breaking-changes.md | New Aspire 13.4 breaking-change scrub list (agent-facing). |
| skills/aspire/evals/eval.yaml | Adds a 13.4 upgrade stimulus to validate specific breaking-change guidance (incl. PG data volumes). |
| evals/csharp-apphost/MyApp.Web/Program.cs | New fixture content used by updated aspireify ServiceDefaults stimulus. |
| evals/csharp-apphost/MyApp.Web/MyApp.Web.csproj | New web project fixture for eval workspace context. |
| evals/csharp-apphost/MyApp.ApiService/Program.cs | New API service fixture for eval workspace context. |
| evals/azure-bicep/infra/main.bicep | New Bicep fixture for IaC→AppHost conversion stimulus. |
Comments suppressed due to low confidence (1)
skills/aspireify/references/typescript-authoring.md:232
- The YARP Routing snippet uses
web.getEndpoint('http')butwebis not declared in the example, making the snippet incomplete.
const api = builder.addProject('api', '../Api/Api.csproj');
const yarp = builder.addYarp('gateway')
.addRoute('/api/{**catch-all}', api.getEndpoint('http'))
.addCatchAllRoute(web.getEndpoint('http'));
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| | `Microsoft.EventHub/namespaces` | `AddAzureEventHubs("eh")` | | | ||
| | `Microsoft.SignalRService/signalR` | `AddAzureSignalR("signalr")` | | | ||
| | `Microsoft.SignalRService/webPubSub` | `AddAzureWebPubSub("wps")` | | | ||
| | `Microsoft.CognitiveServices/accounts` (OpenAI/Foundry) | `AddAzureOpenAI("openai")` / `AddAzureAIFoundry(...)` | confirm name per release | |
Comment on lines
+92
to
+103
| ```ts | ||
| const api = builder.addProject('api', '../Api/Api.csproj') | ||
| .withReference(db) | ||
| .waitFor(db) | ||
| .withExternalHttpEndpoints(); | ||
|
|
||
| builder.addViteApp('web', '../web') | ||
| .withEnvironment('VITE_API_URL', api.getEndpoint('http')) | ||
| .waitFor(api) | ||
| .withExternalHttpEndpoints() | ||
| .withBrowserLogs(); | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Content fixes driven by the vally 0.8.0 eval loop
Stacked on #35 (base branch
dapine/skill-evals-vally). Rebased onto the updated#35 tip, so this branch now inherits the vally 0.8.0 migration and the
scoring: threshold: 0.7gate blocks on every spec (the missing scoring blockswere the earlier eval-CI failure cause).
Each fix follows the eval-first red→green loop: add a capability stimulus that
captures the desired behavior, fix the skill content, and verify the stimulus
passes before committing. Rebase onto
mainonce #35 merges.Issues addressed
b028315awaitnoise in TypeScript AppHost examples — de-await chainableadd*/with*calls across the TS/JS references, add a "when you actually needawait" section + a Hard Rules anti-pattern row, and a fluent-style stimulus.915bbf1shared-buildresource consumerswaitForCompletionon,waitForCompletionvswaitFor, and thetsc --incrementalskip-emit pitfall +prebuildclean fix.6172885d080ff3AddAzure*map,AddBicepTemplatefallback, and the provision-new vs reference-existing decision, with a groundedinfra/main.bicepfixture.6be4a93Verification
vally lint skills→ 6/6 passing; every spec carriesscoring: threshold: 0.7.Capability stimuli are grounded with real workspace fixtures so the executor
activates the skill and reads the relevant reference rather than answering from
memory. Model-backed eval gating runs in CI on vally 0.8.0.
Fixes #31
Fixes #23
Fixes #24
Fixes #29