Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions doctrine/roles/fidelity.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,10 +76,10 @@ screenshots.

**Public-docs silence is not an omission.** When provider docs do not
publish exact field or control labels, and the Dossier (or a setup file)
already records an open question plus a hedge ("the submission control
shown in the console", conceptual field names without invented chrome),
that is correct rendering under I1 — not a research-target blocker
already records a Research limitation plus a hedge ("the submission
control shown in the console", conceptual field names without invented
chrome), that is correct rendering under I1 — not a research-target blocker
demanding a human-verified console capture. Score a nit at most if the
hedge is missing; ensure the open question exists. Demand console capture
hedge is missing; ensure the Research limitation exists. Demand console capture
only when live probing contradicts documented URL/behavior, or when the
operator supplies verified labels in notes.
12 changes: 6 additions & 6 deletions doctrine/roles/review.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ blocker.
**Public-docs silence + hedge.** When public docs do not publish exact
field or control labels, and the guide already hedges ("submission
control shown in the console", conceptual inputs without invented chrome)
with a matching Dossier open question, accept that rendering. Do **not**
with a matching Dossier Research limitation, accept that rendering. Do **not**
raise a blocker demanding a human-verified console capture of unpublished
labels. A missing hedge is a nit (or a `research`-targeted nit to record
the silence); console capture is only for live contradiction of
Expand All @@ -61,9 +61,9 @@ control is named — treat distrust of that citation as out of bounds.
**Speakeasy canonical is fixed.** `doctrine/speakeasy-setup.md` is the fact
ceiling for Speakeasy-side steps. Do not raise blockers that invent
login URLs, catalog-first rewrites, post-credential verification chrome,
or other steps the skeleton does not carry. Gaps in that file are nits
or open questions for a human doctrine edit — never research-target
blockers that expand the guide past the skeleton. Fidelity already fails
or other steps the skeleton does not carry. Gaps in that file are nits for a human doctrine backlog — never operator
questions or research-target blockers that expand the guide past the
skeleton. Fidelity already fails
drift from the skeleton; do not fight that check. When the Dossier (via
Pulse-verified catalog presence, a tenanted-remote override, or
`speakeasy_add_server`) selected only the catalog path or only the
Expand All @@ -76,8 +76,8 @@ requirement.

**Owner-gloss ceiling (when scoring hedges).** Organization-specific
values need at most one obtain-from-owner hedge per section. Re-raising
the same gloss on adjacent fields is a nit. Provider picker enumeration
is a nit or open question, not an achievability blocker.
the same gloss on adjacent fields is a nit. Provider picker enumeration is a nit or Research limitation, not an
operator question or achievability blocker.

## Severity and reporting

Expand Down
35 changes: 18 additions & 17 deletions doctrine/roles/technical-research.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,23 +121,24 @@ path overrides:
- **absent** (auto) — transclude only the Custom remote server path; do
not emit a catalog-presence open question.
- **ambiguous** or **skipped** (or no lookup note), auto, and no
override — keep both add-server bullets and a soft open question for
catalog presence.

## Open questions
Anything you could not confirm from documentation. Flagged, not guessed.
Provider-documented UI (named in prose or shown in screenshots on those
pages) is confirmed — never an open question asking for live console
verification. Reserve this section for silence, unresolved property
conflicts, or live probing that contradicts a documented URL/behavior.
When public docs are silent on exact field or control labels, record the
silence here and leave enough for the Writer to hedge — do not leave the
walkthrough "incomplete" in a way that invites reviewers to demand a
console capture. Your report's `open_questions` must match this section:
do not re-list a UI label already recorded from provider docs as "needs
verification."

This is presentation-only uncertainty, not a scope decision: after a reasonable source search, missing exact UI labels, control names or locations, or equivalent Save/Update/Apply chrome must not produce `awaiting_scope` when the underlying operation and required value are known. Preserve documented identifiers, give the Writer enough evidence for a resilient "visible or equivalent control" hedge, record the silence here, and report status `complete`. Reserve `awaiting_scope` for any unresolved material uncertainty about authentication, endpoints, required credentials, security-sensitive choices, provider capability or feasibility, or conflicting authoritative instructions.
override — keep both add-server bullets and record catalog presence as a research
limitation, not an operator decision.

## Research limitations
Record anything public documentation does not confirm. Flag it, do not guess. This section is provenance for writers and reviewers, not a request for human guidance, and its entries must not be copied into structured `open_questions`. If the operator could only repeat the same public-source search, record a research limitation and continue. Provider-documented UI is confirmed; undocumented labels, incidental pre-filled state, post-save mutability, and later maintenance belong here only when useful to explain a hedge or omission.

## Operator decisions
Open questions are operator-actionable decisions, not a list of documentation gaps. An item may appear here and in structured `open_questions` only when it is material to first connection, cannot be handled with a safe hedge, and answerable from operator knowledge or authority unavailable in public sources. All three conditions are mandatory. Typical valid items are an organization-specific security choice, tenant/region value, or conflicting internal requirement. If none exist, write `None` and return an empty `open_questions` array. Reserve `awaiting_scope` for these decisions only.

Apply these non-question regressions:

- For alternate Configuration and Additional Configuration surfaces, name both documented paths and say to use the one present for the enterprise.
- When a final button label is unpublished, say **Save the integration credentials** without inventing UI chrome.
- Unknown pre-filled redirect URI values do not matter when the reader can add or replace the relevant entry with the documented callback URI.
- If later secret visibility is undocumented, say **Copy the client secret when it is shown** and store it securely.
- Omit whether scopes can be edited after saving because post-save mutability does not affect first connection.

This presentation-only uncertainty must not produce `awaiting_scope`. Preserve documented identifiers, give the Writer enough evidence for resilient wording, record relevant silence under Research limitations, and report status `complete`.

## Provenance
First, the source inventory from the sweep: every documentation property
Expand Down
14 changes: 4 additions & 10 deletions doctrine/roles/writer.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,7 @@ recovery (reset a secret next month, rotate for drift), ongoing admin
surfaces — stay out of the setup files. Guides cover getting the server
working, not maintenance.

If the Dossier is missing something you need — a step you cannot render
without inventing, a term you cannot define from recorded facts — stop
writing around it and report it as an open question in your structured
report. A visible gap is a finding for the pipeline; an invisible patch is
a defect.
If the Dossier is missing something needed for first connection, do not invent it. Return a structured open question only when the gap is material to first connection, cannot be handled with a safe hedge, and is answerable from operator knowledge or authority unavailable in public sources. If the operator could only repeat public research, treat the gap as a research limitation: use Dossier-backed resilient wording or omit irrelevant detail and continue. A hidden guess is still a defect.

## Setup grammar (the parts that bite)

Expand Down Expand Up @@ -142,13 +138,11 @@ Dossier.

Treat Dossier-listed presentation-only uncertainty as renderable, not blocking: when the underlying operation and required value are known but an exact UI label, control name or location, or Save/Update/Apply variant is not, preserve documented identifiers and use a resilient "visible or equivalent control" hedge. Do not return an open question or incomplete status solely for that uncertainty.

Dossier Research limitations are not operator questions. Render around them without returning them. Only Dossier Operator decisions or newly discovered gaps that pass the same three-part operator-actionability test belong in structured `open_questions`.

Status `ok` when `external.md` and `speakeasy.md` exist on disk, are
complete, and every fact traces to the Dossier; status `blocked` only when
Dossier gaps make the guide unwritable. A structured report without those
files is incomplete. List open questions either way — but only gaps the
Dossier does not already record. Rendering around a Dossier-listed open
question is expected work, not a new question; restating it doubles the
human's
checklist. If rendering changed the picture (a workaround you chose, a
files is incomplete. If rendering changed the picture (a workaround you chose, a
fallback the reader needs verified), put that in `notes`, not in a
duplicate question.
6 changes: 3 additions & 3 deletions factory/coordinator.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ Accept only the exact keys `slug`, `stage`, and `artifacts`, with string slug/st

Begin Phase 1 with one caught boundary executing exactly `bash factory/scripts/inspect-inputs.sh /input/issue.json /input/catalog.json`. Read issue evidence, catalog identity fields, available personas, and existing guide slugs only from that command's JSON output; do not construct another initial-inspection tool program or read either raw input another way. Resolve exactly one provider and lowercase kebab-case slug from this output. Prefer an existing slug on a confident match; never create an alias duplicate. If provider/slug is missing, conflicting, or ambiguous, choose `blocked`, leave all three identity fields null, and report without guide edits.

For a resolved slug, execute exactly `bash factory/scripts/inspect-guide-context.sh <slug>` in one caught boundary. Read authority, role, schema, persona, representative-guide, and target-artifact contents only from its `.files` object. If Kit spills this large result, permit only bounded reads of the helper-generated spill artifact until every `.files` entry is consumed; use only installed jq and sed for spill reads; Python is unavailable; never inspect repository files directly or run another Phase 1 file-discovery tool. Resolve the persona only after reading every available definition under `doctrine/personas/`: default to `it-admin`; override it only when the issue confidently names an available repository persona. Pass the selected `doctrine/personas/<persona>.md` file to every downstream agent and reviewer. Resolve catalog presence only from the `.catalog` object returned by the initial command; never inspect `/input/catalog.json` directly. Preserve tenanted remote and `speakeasy_add_server` catalog/custom-remote doctrine. Skipped, malformed, stale, or ambiguous lookup means unknown and an open question, never absence.
For a resolved slug, execute exactly `bash factory/scripts/inspect-guide-context.sh <slug>` in one caught boundary. Read authority, role, schema, persona, representative-guide, and target-artifact contents only from its `.files` object. If Kit spills this large result, permit only bounded reads of the helper-generated spill artifact until every `.files` entry is consumed; use only installed jq and sed for spill reads; Python is unavailable; never inspect repository files directly or run another Phase 1 file-discovery tool. Resolve the persona only after reading every available definition under `doctrine/personas/`: default to `it-admin`; override it only when the issue confidently names an available repository persona. Pass the selected `doctrine/personas/<persona>.md` file to every downstream agent and reviewer. Resolve catalog presence only from the `.catalog` object returned by the initial command; never inspect `/input/catalog.json` directly. Preserve tenanted remote and `speakeasy_add_server` catalog/custom-remote doctrine. Skipped, malformed, stale, or ambiguous lookup means unknown, never absence; preserve both safe setup paths and record a research limitation rather than asking the operator to repeat the lookup.

## Phase 2 — research and scope gate

Scope-gate classification is strict. Material uncertainty is limited to an unknown authentication model, endpoint, required credential, security-sensitive operator choice, provider capability or feasibility, or an unresolved authoritative-source conflict that could make the guide unsafe or unusable. Presentation-only uncertainty never selects `awaiting_scope`. Missing exact UI labels, control names or locations, and equivalent Save/Update/Apply chrome are presentation-only when the underlying operation and required value are known. After a reasonable source search, preserve documented identifiers, write a resilient hedge such as the visible or equivalent control, record the documentation silence, and continue. Open questions alone do not select `awaiting_scope`; classify each by its effect on safe first connection.
Scope-gate classification is strict. Material uncertainty is limited to an unknown authentication model, endpoint, required credential, security-sensitive operator choice, provider capability or feasibility, or an unresolved authoritative-source conflict that could make the guide unsafe or unusable. Presentation-only uncertainty never selects `awaiting_scope`. Missing exact UI labels, control names or locations, and equivalent Save/Update/Apply chrome are presentation-only when the underlying operation and required value are known. After a reasonable source search, preserve documented identifiers, write a resilient hedge such as the visible or equivalent control, record the documentation silence, and continue. Open questions are operator-actionable decisions, not a list of documentation gaps. Each must be material to first connection, cannot be handled with a safe hedge, and answerable from operator knowledge or authority unavailable in public sources. If the operator could only repeat the same public-source search, record a research limitation and continue. Only an item passing all three gates may select `awaiting_scope` or enter any structured `open_questions` array.

Start the technical-research subagent in a caught boundary with the selected persona file, authority files, resolved identity/catalog facts, issue evidence, existing artifacts, primary-source requirement, and write access only to `research.md` and `meta.yaml`. Set `output_schema` to the exact `factory/schemas/research-status.schema.json`. Apply the universal transport/schema check and one-repair limit. In another caught file-validation boundary, execute exactly `bash factory/scripts/inspect-guide-artifacts.sh <slug> research` and confirm its result agrees with the valid output. Only material unanswered decisions select `awaiting_scope`; authoritative evidence blockers select `blocked`; operational/caught errors select `failed`. Each terminal state skips later model phases and reaches reporting.

Expand All @@ -55,7 +55,7 @@ Deterministic scenario rulings: failed reviewer output -> `failed`, zero increme

## Phase 5 — strict atomic report (always runs)

This phase is cleanup/finalization, not a model phase, and runs even when `stop_model_phases` is true. Create `/workspace/.factory` in a caught boundary. Construct a strict `factory/schemas/run-report.schema.json` value reflecting terminal state, physical durable artifacts, open questions, blockers, nits, and actual completed-wave count. Failed reports list no exported artifacts, per schema.
This phase is cleanup/finalization, not a model phase, and runs even when `stop_model_phases` is true. Create `/workspace/.factory` in a caught boundary. Construct a strict `factory/schemas/run-report.schema.json` value reflecting terminal state, physical durable artifacts, operator-actionable open questions, blockers, nits, and actual completed-wave count. Research limitations never enter the run report; `open_questions` is empty unless an item passes the Phase 2 three-part gate. Failed reports list no exported artifacts, per schema.

Ordering is mandatory:

Expand Down
1 change: 0 additions & 1 deletion factory/scripts/publish.sh
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,6 @@ render_report_comment() {
"", "### Summary", "", (.summary|bound), ""]
+ (if .outcome == "awaiting_scope" then (.open_questions|items("### Material decisions"; true)) else [] end)
+ (.blockers|items("### Blockers"; false))
+ (if .outcome == "awaiting_scope" then [] else (.open_questions|items("### Open questions"; false)) end)
+ (.nits|items("### Nits"; false))
+ [if .outcome == "converged" then "Ready for review."
elif .outcome == "awaiting_scope" then "Reply with the numbered decisions, then re-add `guide:draft`."
Expand Down
15 changes: 12 additions & 3 deletions factory/tests/test-coordinator.sh
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,9 @@ for phrase in \
'outside /workspace/guides/<slug>' \
"Presentation-only uncertainty never selects \`awaiting_scope\`" \
'Missing exact UI labels, control names or locations, and equivalent Save/Update/Apply chrome are presentation-only' \
"Open questions alone do not select \`awaiting_scope\`" \
'Open questions are operator-actionable decisions, not a list of documentation gaps' \
'If the operator could only repeat the same public-source search, record a research limitation and continue' \
'material to first connection, cannot be handled with a safe hedge, and answerable from operator knowledge or authority' \
'Revision agents must not run validation commands' \
"including \`go\`, \`go run\`, \`npx\`, Python, and \`/usr/local/bin/lint-guide\`"; do
grep -Fq "$phrase" "$CONTRACT" || fail "missing contract: $phrase"
Expand Down Expand Up @@ -103,8 +105,15 @@ for role_contract in doctrine/roles/technical-research.md doctrine/roles/writer.
fail "missing presentation-only uncertainty policy: $role_contract"
done

grep -Fq 'any unresolved material uncertainty' "$ROOT/doctrine/roles/technical-research.md" ||
fail 'research role narrows material uncertainty to operator decisions'
for example in \
'alternate Configuration and Additional Configuration surfaces' \
'Save the integration credentials' \
'pre-filled redirect URI values' \
'Copy the client secret when it is shown' \
'whether scopes can be edited after saving'; do
grep -Fq "$example" "$ROOT/doctrine/roles/technical-research.md" ||
fail "missing non-question regression example: $example"
done

research_line="$(grep -n 'technical-research subagent' "$CONTRACT" | head -1 | cut -d: -f1)"
persona_line="$(grep -n 'Resolve the persona only after' "$CONTRACT" | head -1 | cut -d: -f1)"
Expand Down
Loading
Loading