Skip to content

Surface guide factory review on the issue #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
walker-tx merged 2 commits into from
Jul 23, 2026
Merged

Surface guide factory review on the issue #9

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
60e8616
Surface pipeline blockers on the issue and keep unconverged drafts.
walker-tx Jul 23, 2026
d6d1091
Make pipeline review comments decision-shaped, not jargon dumps.
walker-tx Jul 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 93 additions & 8 deletions .github/workflows/guide-draft.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,17 @@ jobs:
echo "CURSOR_API_KEY secret is not set" > "${RUNNER_TEMP}/failure_reason.txt"
exit 1
fi
# Fold issue comments into distill context so clarifications on the
# ticket (not only the body) reach --notes on retry.
comments=$(gh api "repos/${GH_REPO}/issues/${ISSUE_NUMBER}/comments" \
--jq '[.[].body] | join("\n\n---\n\n")' \
2>/dev/null || echo "")
if [ -n "${comments}" ]; then
export ISSUE_BODY="${ISSUE_BODY:-}

## Issue thread (for clarifications)
${comments}"
fi
set +e
npm run resolve-issue -- --output "${RUNNER_TEMP}/resolved.json"
code=$?
Expand Down Expand Up @@ -149,7 +160,7 @@ jobs:
echo "**Candidates:** ${candidates}"
fi
echo
echo "Edit the issue (or reply clarifying which MCP server), then re-add \`guide:draft\`."
echo "Reply on this issue (or edit the body) clarifying which MCP server, then re-add \`guide:draft\`."
} > "${RUNNER_TEMP}/failure_reason.txt"
exit 1
fi
Expand Down Expand Up @@ -214,10 +225,30 @@ jobs:
npm run draft-guide -- "${SLUG}" "${args[@]}"
code=$?
set -e
if [ "$code" -ne 0 ]; then
echo "draft-guide exited ${code} (see workflow logs). Converged drafts still open a PR; unconverged/blocked/failed fail the job." > "${RUNNER_TEMP}/failure_reason.txt"
exit "$code"
# Prefer the newest run record for this slug (written even on unconverged).
record=""
if compgen -G "${GITHUB_WORKSPACE}/retro/runs/*-${SLUG}.json" > /dev/null; then
record=$(ls -t "${GITHUB_WORKSPACE}"/retro/runs/*-"${SLUG}".json | head -n1)
echo "record=${record}" >> "$GITHUB_OUTPUT"
cp "$record" "${RUNNER_TEMP}/run-record.json"
fi
# CLI: 0 = converged, 2 = unconverged/blocked/failed guide status,
# 1 = hard failure. Exit 2 still wrote files — open a PR for humans.
if [ "$code" -eq 0 ]; then
echo "outcome=converged" >> "$GITHUB_OUTPUT"
exit 0
fi
if [ "$code" -eq 2 ]; then
if [ -d "${GITHUB_WORKSPACE}/guides/${SLUG}" ]; then
echo "outcome=unconverged" >> "$GITHUB_OUTPUT"
echo "draft-guide exited 2 (unconverged/blocked/failed). Opening a draft PR with whatever was written for human review." > "${RUNNER_TEMP}/failure_reason.txt"
exit 0
fi
echo "draft-guide exited 2 and guides/${SLUG}/ is missing" > "${RUNNER_TEMP}/failure_reason.txt"
exit 1
fi
echo "draft-guide exited ${code} (hard failure; see workflow logs)" > "${RUNNER_TEMP}/failure_reason.txt"
exit "$code"

- name: Commit and push
if: steps.preflight.outputs.refused != 'true' && success()
Expand Down Expand Up @@ -245,47 +276,101 @@ jobs:
BRANCH: ${{ steps.branch.outputs.name }}
SLUG: ${{ steps.distill.outputs.slug }}
PROVIDER: ${{ steps.distill.outputs.provider }}
OUTCOME: ${{ steps.draft.outputs.outcome }}
run: |
set -euo pipefail
title="Draft guide: ${PROVIDER} (#${ISSUE_NUMBER})"
if [ "${OUTCOME}" = "unconverged" ]; then
title="Draft guide (unconverged): ${PROVIDER} (#${ISSUE_NUMBER})"
fi
title="${title:0:256}"
body_file="${RUNNER_TEMP}/pr-body.md"
{
echo "Closes #${ISSUE_NUMBER}"
echo
echo "Factory draft of \`guides/${SLUG}/\` via \`draft-guide\` (Cursor SDK)."
echo
if [ "${OUTCOME}" = "unconverged" ]; then
echo "**Pipeline status:** unconverged (reviewers still had blockers after max rounds)."
echo "See the issue comment **Pipeline review** for unresolved blockers and open questions."
echo "Do not merge until those are settled."
echo
fi
echo "Human review still required — agents never commit; this Action did."
} > "$body_file"
# Append a short blocker summary into the PR body when present.
if [ -f "${RUNNER_TEMP}/run-record.json" ]; then
echo >> "$body_file"
bash "${GITHUB_WORKSPACE}/scripts/ci/format-pipeline-review.sh" \
"${RUNNER_TEMP}/run-record.json" "" \
"${GITHUB_WORKSPACE}/guides/${SLUG}" >> "$body_file"
fi
pr_url=$(gh pr create --draft --base main --head "$BRANCH" --title "$title" --body-file "$body_file" | tail -n1)
echo "pr_url=$pr_url" >> "$GITHUB_OUTPUT"

- name: Comment PR URL
- name: Comment pipeline review on issue
if: steps.preflight.outputs.refused != 'true' && success()
env:
PR_URL: ${{ steps.open_pr.outputs.pr_url }}
OUTCOME: ${{ steps.draft.outputs.outcome }}
SLUG: ${{ steps.distill.outputs.slug }}
run: |
gh issue comment "$ISSUE_NUMBER" --body "Draft PR opened: ${PR_URL}"
set -euo pipefail
body_file="${RUNNER_TEMP}/pipeline-review-comment.md"
if [ -f "${RUNNER_TEMP}/run-record.json" ]; then
bash "${GITHUB_WORKSPACE}/scripts/ci/format-pipeline-review.sh" \
"${RUNNER_TEMP}/run-record.json" "${PR_URL}" \
"${GITHUB_WORKSPACE}/guides/${SLUG}" > "$body_file"
else
{
echo "## Pipeline review"
echo
if [ "${OUTCOME}" = "unconverged" ]; then
echo "Draft PR opened (pipeline **unconverged**): ${PR_URL}"
else
echo "Draft PR opened: ${PR_URL}"
fi
echo
echo "_No run record found to summarize blockers / open questions._"
} > "$body_file"
fi
gh issue comment "$ISSUE_NUMBER" --body-file "$body_file"

- name: Mark blocked on failure
if: steps.preflight.outputs.refused != 'true' && failure()
env:
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
SLUG: ${{ steps.distill.outputs.slug }}
run: |
reason="(no reason file written; check workflow logs)"
if [ -f "${RUNNER_TEMP}/failure_reason.txt" ]; then
reason=$(cat "${RUNNER_TEMP}/failure_reason.txt")
fi
body_file="${RUNNER_TEMP}/failure-comment.md"
{
echo "\`guide:draft\` run failed."
echo "\`guide:draft\` run failed (no PR opened)."
echo
echo "$reason"
echo
echo "**Workflow run:** ${RUN_URL}"
echo
echo "Re-add \`guide:draft\` to retry after clarifying or fixing."
} > "$body_file"
# If a run record exists from a partial draft, surface blockers here too.
if [ -f "${RUNNER_TEMP}/run-record.json" ]; then
echo >> "$body_file"
bash "${GITHUB_WORKSPACE}/scripts/ci/format-pipeline-review.sh" \
"${RUNNER_TEMP}/run-record.json" "" \
"${GITHUB_WORKSPACE}/guides/${SLUG}" >> "$body_file" || true
elif [ -n "${SLUG}" ] && compgen -G "${GITHUB_WORKSPACE}/retro/runs/*-${SLUG}.json" > /dev/null; then
record=$(ls -t "${GITHUB_WORKSPACE}"/retro/runs/*-"${SLUG}".json | head -n1)
echo >> "$body_file"
bash "${GITHUB_WORKSPACE}/scripts/ci/format-pipeline-review.sh" \
"$record" "" "${GITHUB_WORKSPACE}/guides/${SLUG}" >> "$body_file" || true
else
{
echo "Reply on this issue with clarifications, then re-add \`guide:draft\`."
} >> "$body_file"
fi
gh issue edit "$ISSUE_NUMBER" --add-label "guide:blocked" || true
gh issue comment "$ISSUE_NUMBER" --body-file "$body_file"

Expand Down
51 changes: 36 additions & 15 deletions docs/agents/guide-factory.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ commits, pushes, and opens the PR.

Workflow: [`.github/workflows/guide-draft.yml`](../../.github/workflows/guide-draft.yml).
Distill CLI: `npm run resolve-issue` in `scripts/cursor-sdk/`.
Review comment formatter: [`scripts/ci/format-pipeline-review.sh`](../../scripts/ci/format-pipeline-review.sh).

## How to file an issue

Expand All @@ -16,13 +17,29 @@ Distill CLI: `npm run resolve-issue` in `scripts/cursor-sdk/`.
- `Draft a BigQuery MCP setup guide`
- `We need HubSpot — prefer OAuth docs at https://…`
2. Add the label `guide:draft`.
3. Wait for the Action. On success you get a draft PR (`Closes #<issue>`).
On clarification or failure you get `guide:blocked` and a comment; edit the
issue and re-add `guide:draft` to retry.
3. Wait for the Action. You get:
- a **Resolved as …** comment (distill intent),
- a **Pipeline review** comment (unresolved blockers, open questions, nits),
- a draft PR (`Closes #<issue>`) when files were written — including
**unconverged** runs (PR title/body say so).

Persona defaults to `it-admin` unless the distill step confidently finds a
known id under `docs/personas/`.

## Where to put clarifications

When the **Pipeline review** comment asks for a call (exact UI labels, whether
to drop a recovery branch, etc.):

1. **Reply on the issue** with the answers (preferred — easy to skim in the
thread), and/or
2. **Edit the issue body** with the same facts.

Then re-add `guide:draft`. Distill re-reads the **body and the full comment
thread** into `--notes` for the next run. You do **not** need to paste the
whole guide into the ticket — answer the open questions / blockers listed in
the review comment.

## Labels

The workflow **creates these labels automatically** if missing. You can still
Expand All @@ -32,7 +49,7 @@ create them by hand for triage before the first run:
| --- | --- |
| `guide:draft` | Trigger — removed as soon as the job accepts the work |
| `guide:in-progress` | Set while distill + pipeline run; always cleared in `always()` |
| `guide:blocked` | Set on preflight refusal, distill clarification, or pipeline failure; cleared when a new successful accept starts |
| `guide:blocked` | Set on preflight refusal, distill clarification, or hard failure; cleared when a new successful accept starts |

Suggested colors (optional): draft = blue, in-progress = yellow, blocked = red.

Expand All @@ -51,18 +68,22 @@ pushing with full permissions matters.

1. **Preflight** — refuse if an open collaborator PR already `Closes #N`.
2. **Labels** — remove `guide:draft` + `guide:blocked`, add `guide:in-progress`.
3. **Distill** — light Cursor agent (`composer-2.5` / `CURSOR_MODEL_LIGHT`)
reads title+body (+ existing `guides/*` slugs) and writes structured JSON
(`slug`, `provider`, `persona`, `notes`) or `needs_clarification`.
4. **Comment** — on `ok`, comment a short “Resolved as `slug` …” summary.
5. **Draft** — `npm run draft-guide -- <slug> --overwrite [--notes …] [--persona …]`.
3. **Distill** — light Cursor agent reads title + body + issue comments (+
existing `guides/*` slugs) → structured JSON or `needs_clarification`.
4. **Comment** — “Resolved as `slug` …” summary.
5. **Draft** — `npm run draft-guide -- <slug> --overwrite [--notes …]`.
6. **PR** — commit `guides/<slug>/` + matching `retro/runs/*-<slug>.json`,
push `guide/issue-<N>-<slug>`, open a **draft** PR.
7. **Failure** — `guide:blocked` + comment with reason and Actions run URL.
8. **Always** — remove `guide:in-progress`.

CLI exit `2` (unconverged / blocked / failed) fails the job so the blocked
path runs. Only converged drafts open a PR for human review.
push `guide/issue-<N>-<slug>`, open a **draft** PR (also on unconverged
when files exist).
7. **Comment** — **Pipeline review** on the issue (blockers / open questions /
nits + PR link). Same summary is appended to the PR body.
8. **Hard failure** — `guide:blocked` + comment (includes review summary when
a run record exists).
9. **Always** — remove `guide:in-progress`.

CLI exit `0` (converged) and exit `2` (unconverged / blocked / failed guide
status with files on disk) both open a draft PR. Hard failures (exit `1`,
missing `guides/<slug>/`) take the blocked path with no PR.

## What v1 does not do

Expand Down
Loading