Skip to content

Fix dead slash-command refs and cookbook doc/YAML drift - #109

Open
peterzhang12312 wants to merge 2 commits into
anthropics:mainfrom
peterzhang12312:fix/dead-skill-refs-and-cookbook-doc-drift
Open

Fix dead slash-command refs and cookbook doc/YAML drift#109
peterzhang12312 wants to merge 2 commits into
anthropics:mainfrom
peterzhang12312:fix/dead-skill-refs-and-cookbook-doc-drift

Conversation

@peterzhang12312

Copy link
Copy Markdown

Summary

The documented validation suite passes clean on main, but two defect classes sit outside what any of it checks. Both are user-visible.

Dead slash-command references

CLAUDE.md requires slash-command names in prose to be the actual skills/<name>/ directory name — short forms "look right in prose but are dead commands." claude plugin validate treats /foo in prose as text, so these shipped silently:

  • ip-legal/skills/cold-start-interview — 11 refs across 6 lines used short forms: /cd (→ cease-desist), /fto (→ fto-triage), /infringe (→ infringement-triage), /oss (→ oss-review), /clause (→ ip-clause-review). These are the lines that tell the user which skills their answers light up, so every name in them is one the user may type.
  • legal-clinic/README.md/legal-clinicgrations, a mangled find-and-replace. The intact form of the identical paragraph in product-legal/README.md gives the correct text.
  • litigation-legal/skills/matter-intake`/matter` briefing`/matter-briefing`.
  • product-legal/skills/customize/ai-governance-legal addressed a plugin as a command; reworded to name the plugin.
  • Three command names were line-wrapped inside their backticks (`/handbook-\nupdates`), so they render with a space and are not typable as written. Reflowed.

Cookbook README security tables contradicting their own YAML

Cookbook rule 2 requires the README security table and agent.yaml comments to match what the YAML grants. lint-tool-scope.py only enforces rule 1 and never opens subagents/*.yaml or README.md, so rule 2 was unenforced:

  • launch-radar README claimed the orchestrator holds Linear/Jira/Asana/Drive. It grants no mcp_toolset at all — and the adjacent agent.yaml comment said so. A reader auditing blast radius from the README would overestimate it; a reader "fixing" the YAML to match would break rule 1.
  • docket-watcher and reg-monitor READMEs advertised a gdrive connector on components that grant none. docket-watcher's own README refuted its table three lines later.
  • diligence-grid claimed iManage read access without noting it ships disabled.
  • Five MCP servers were declared in mcp_servers but granted to no component (definely, gdrive×3, docusign), with deploy instructions telling operators to export URLs for connectors nothing could reach. Removed the declarations and the env-var docs rather than widening any subagent's scope. Nothing in scripts/ or .github/ referenced them.
  • Two agent.yaml comments claimed leaves hold web_fetch that no leaf has (copy-paste from reg-monitor, the one cookbook where it is true).
  • Two adaptation notes pointed at agent.yaml for an iManage toggle that lives in a subagent; following them literally would have added an MCP grant to an orchestrator and broken rule 1.
  • Split the combined <leaf> / Orchestrator rows. That row published a union of two tiers, so it over-claimed Glob for three leaves and Agent for every leaf (all have callable_agents: []), and made the row uncheckable against either component.

No under-disclosure was found: every enabled tool and connector, including feed-reader's web_fetch and each Write-holder's exact toolset, was already disclosed. All defects ran in the over-claim direction.

Regression cover

Both classes now fail CI instead of shipping:

  • scripts/lint-skill-refs.py — every /foo and /plugin:skill in plugin prose resolves to a real skill directory; also catches names line-wrapped inside backticks. Built-ins are exempt.
  • scripts/lint-cookbook-docs.py — README security-table tools and connectors match the YAML in both directions, declared MCP servers are granted somewhere, agent.yaml comments are true, and combined rows are rejected.

Verified both scripts reproduce the full hand-found defect set against the pre-fix tree and pass against this one.

Also corrected the JSON sanity command in CLAUDE.md: glob('**/*.json') does not match dotfiles, so it was checking 15 files and skipping all 27 that live under .claude-plugin/ or are named .mcp.json.

Left alone per CLAUDE.md: the I1 curated sort order, the expected plugin-root CLAUDE.md template warnings, per-plugin .gitignore differences, the two missing hooks/hooks.json, the references/ path gap, and vendor-authored content under external_plugins/.

Test plan

  • python scripts/lint-skill-refs.py — passes on patched tree (13 plugins / 151 skills)
  • python scripts/lint-cookbook-docs.py — passes on patched tree (5/5 cookbooks)
  • python scripts/lint-tool-scope.py — passes on patched tree (5/5 cookbooks)
  • claude plugin validate .claude-plugin/marketplace.json — passes
  • Both new lint scripts verified to reproduce the full hand-found defect set against the pre-fix tree (not passing vacuously)

The documented validation suite passes clean on main, but two defect classes
sit outside what any of it checks. Both are user-visible.

## Dead slash-command references

CLAUDE.md requires slash-command names in prose to be the actual
`skills/<name>/` directory name — short forms "look right in prose but are
dead commands." `claude plugin validate` treats `/foo` in prose as text, so
these shipped silently:

- `ip-legal/skills/cold-start-interview` — 11 refs across 6 lines used short
  forms: `/cd` (→ `cease-desist`), `/fto` (→ `fto-triage`), `/infringe`
  (→ `infringement-triage`), `/oss` (→ `oss-review`), `/clause`
  (→ `ip-clause-review`). These are the lines that tell the user which skills
  their answers light up, so every name in them is one the user may type.
- `legal-clinic/README.md` — `/legal-clinicgrations`, a mangled
  find-and-replace. The intact form of the identical paragraph in
  `product-legal/README.md` gives the correct text.
- `litigation-legal/skills/matter-intake` — `` `/matter` briefing `` →
  `` `/matter-briefing` ``.
- `product-legal/skills/customize` — `/ai-governance-legal` addressed a plugin
  as a command; reworded to name the plugin.
- Three command names were line-wrapped inside their backticks
  (`` `/handbook-\nupdates` ``), so they render with a space and are not
  typable as written. Reflowed.

## Cookbook README security tables contradicting their own YAML

Cookbook rule 2 requires the README security table and `agent.yaml` comments
to match what the YAML grants. `lint-tool-scope.py` only enforces rule 1 and
never opens `subagents/*.yaml` or `README.md`, so rule 2 was unenforced:

- `launch-radar` README claimed the orchestrator holds Linear/Jira/Asana/Drive.
  It grants no `mcp_toolset` at all — and the adjacent `agent.yaml` comment
  said so. A reader auditing blast radius from the README would overestimate
  it; a reader "fixing" the YAML to match would break rule 1.
- `docket-watcher` and `reg-monitor` READMEs advertised a gdrive connector on
  components that grant none. `docket-watcher`'s own README refuted its table
  three lines later.
- `diligence-grid` claimed iManage read access without noting it ships
  disabled.
- Five MCP servers were declared in `mcp_servers` but granted to no component
  (`definely`, `gdrive`×3, `docusign`), with deploy instructions telling
  operators to export URLs for connectors nothing could reach. Removed the
  declarations and the env-var docs rather than widening any subagent's scope.
  Nothing in `scripts/` or `.github/` referenced them.
- Two `agent.yaml` comments claimed leaves hold `web_fetch` that no leaf has
  (copy-paste from `reg-monitor`, the one cookbook where it is true).
- Two adaptation notes pointed at `agent.yaml` for an iManage toggle that
  lives in a subagent; following them literally would have added an MCP grant
  to an orchestrator and broken rule 1.
- Split the combined `<leaf> / Orchestrator` rows. That row published a union
  of two tiers, so it over-claimed `Glob` for three leaves and `Agent` for
  every leaf (all have `callable_agents: []`), and made the row uncheckable
  against either component.

No under-disclosure was found: every enabled tool and connector, including
`feed-reader`'s `web_fetch` and each Write-holder's exact toolset, was already
disclosed. All defects ran in the over-claim direction.

## Regression cover

Both classes now fail CI instead of shipping:

- `scripts/lint-skill-refs.py` — every `/foo` and `/plugin:skill` in plugin
  prose resolves to a real skill directory; also catches names line-wrapped
  inside backticks. Built-ins are exempt.
- `scripts/lint-cookbook-docs.py` — README security-table tools and connectors
  match the YAML in both directions, declared MCP servers are granted
  somewhere, `agent.yaml` comments are true, and combined rows are rejected.

Verified both scripts reproduce the full hand-found defect set against the
pre-fix tree and pass against this one.

Also corrected the JSON sanity command in CLAUDE.md: `glob('**/*.json')` does
not match dotfiles, so it was checking 15 files and skipping all 27 that live
under `.claude-plugin/` or are named `.mcp.json`.

Left alone per CLAUDE.md: the I1 curated sort order, the expected plugin-root
CLAUDE.md template warnings, per-plugin `.gitignore` differences, the two
missing `hooks/hooks.json`, the `references/` path gap, and vendor-authored
content under `external_plugins/`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vz1DwCuBbxbqkG6fBGBZp9
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown

Caution

Pull Request opener is not an author or co-author of any commit in this PR.

This check is blocked to guard against commits being submitted under a trusted identity the submitter does not control. If this PR is a legitimate cherry-pick, release-engineering submission, or mailing-list-style patch delivery, the repository maintainer can opt out of this check by setting require-opener-as-author: 'false' on the CLA-assistant step in the repository's workflow.


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


0 out of 2 committers have signed the CLA.
@peterzhang12312
❌ @peterzhang12312-jpg

Warning

1 commit in this PR was authored by an email address that is not linked to any GitHub user, so we cannot tell whether the author has signed the CLA.

Unlinked author:

  • peterzhang12312-jpg <your-email@example.com>

To unblock this PR, do one of the following:

  1. Link the email to your GitHub account (recommended). Add each address above at github.com/settings/emails, then push another commit (or comment recheck) so this check re-runs. See why commits are not linked to a user for details.

  2. Rewrite the commits to use an email that is already linked to your GitHub account:

    # Set the correct email locally (one-off, for this repo):
    git config user.email you@example.com
    # Rewrite every commit on this branch with the corrected identity:
    git rebase -i --root --exec 'git commit --amend --reset-author --no-edit'
    git push --force-with-lease

    After the push, comment recheck on this PR (or just re-push) to re-run the check.

    You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

Repo text is UTF-8 and the lint scripts print non-ASCII markers (✓, —), but
Windows consoles default to cp1252, so print() raised UnicodeEncodeError and
lint-cookbook-docs.py's bare read_text()/open() calls raised
UnicodeDecodeError on the same files. Reconfigure stdout/stderr to UTF-8 in
each script and pass encoding="utf-8" explicitly on every file read, instead
of requiring every caller to remember PYTHONUTF8=1.

Verified: all three scripts pass with no env override and under
PYTHONIOENCODING=cp1252 forced.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants