Harden non-interactive Aspire scaffold guidance and init eval graders

[David Pine (IEvangelist)]

Summary

Rebased onto latest main and trimmed to the delta that main's vally 0.8.0 eval migration (#35) did not already cover. #35 reworded the hang-prone routing prompts (should_trigger_*) into "How do I…?" questions for the same reason this PR originally targeted — so the executor explains the command instead of running a scaffold that hangs (aspire#11595). This PR now adds only what's still missing:

• Docs: document the still-open aspire new install hang (aspire#11595) and the now-fixed Unix stdin block (aspire#16791) in the aspire-init troubleshooting table, and require explicit --non-interactive (plus --language, --name, --output, and template-specific flags) in the agent/CI guidance (SKILL.md, templates.md, init-workflow.md, orchestration app-commands.md).
• Evals: add non-interactive command graders to the aspire-init core-flow stimuli (init-existing-001, init-new-001, init-py-001) — which Evals: vally 0.8.0 skills migration + comparative baselines + docs #35 left untouched — asserting the skill emits --non-interactive/--language/--name/--output and avoids unsupported flags (--yes, --features, --skip-install), and reword those prompts to explain rather than execute.
• CI: add job-level timeout-minutes ceilings to the CI and nightly eval steps so a hung subprocess can't hold a runner indefinitely.

Dropped from the original PR (superseded by #35)

• The routing prompt rewrites for should_trigger_*, should_not_trigger_08, and should_trigger_09 — main already reworded these.
• The per-run --timeout 2m, which would override the deliberate 600s per-trial budget Evals: vally 0.8.0 skills migration + comparative baselines + docs #35 introduced (and reintroduce the hard-fail it avoids).

Upstream context

• aspire new hangs on package install aspire#11595 — still open (aspire new package-install hang)
• aspire init/add/restore can hang silently when stdin is a TTY (macOS) aspire#16791 — fixed/closed 2026-08-11 (Unix stdin TTY hang)

Validation

• vally lint skills → 6/6 passed
• vally lint --eval-spec skills/aspire-init/evals/eval.yaml → 0 errors (10 pre-existing warnings, unchanged from main)

[Copilot]

Pull request overview

This PR hardens Aspire skill routing evals and CI guidance to avoid hangs caused by interactive/stalled aspire new / aspire init subprocesses, while clarifying the supported non-interactive command forms for automation.

Changes:

• Updates routing and workflow eval stimuli to require explicit --non-interactive (and --language for aspire init) and to discourage unsupported flags like --yes.
• Expands Aspire init/new documentation with safe non-interactive scaffolding guidance, known upstream hang context, and a Unix stdin workaround.
• Adds workflow step and per-eval timeouts to prevent stalled eval runs from holding CI indefinitely.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
skills/aspireify/evals/eval.yaml	Refines routing stimulus to explicitly reference non-interactive aspire init without running commands.
skills/aspire/evals/eval.yaml	Adjusts routing stimulus to explicitly test ownership of non-interactive aspire init.
skills/aspire-orchestration/references/app-commands.md	Updates recommended aspire new/init command forms for agent/CI non-interactive execution.
skills/aspire-init/SKILL.md	Strengthens non-interactive scaffolding guidance and documents known hang/timeout handling.
skills/aspire-init/references/templates.md	Clarifies automation-safe template invocation requirements and expands examples.
skills/aspire-init/references/init-workflow.md	Makes the non-interactive aspire init form explicit in documentation.
skills/aspire-init/evals/eval.yaml	Updates core-flow and routing eval prompts/graders to enforce supported non-interactive options.
.github/workflows/skill-eval.yml	Adds step-level timeout and per-run vally timeout to bound hangs in CI gate evals.
.github/workflows/skill-eval-nightly.yml	Adds step-level timeout and per-run vally timeout to bound hangs in nightly evals.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.