Skip to content

NLPM Audit: 4 bugs and 3 security improvements found (score: 83/100) #84

@xiaolai

Description

@xiaolai

Automated audit: This issue was generated by NLPM, a natural language programming linter, running via claude-code-action. Please evaluate the findings on their merits.

About This Audit

Agent Skills for Context Engineering is a high-quality collection — it scored 83/100 on NLPM's natural-language programming rubric, with 12 of the 14 core skills earning 90 or above. This audit surfaces the few actionable issues worth fixing; everything else is strong work.

NLPM is a 100-point quality scoring tool for Claude Code NL artifacts (skills, agents, hooks). It applies deterministic penalties against a rulebook and flags bugs that affect discoverability or correctness. An automated pipeline then opens PRs for verified bugs.


Bugs Found (PR-worthy)

These are structural bugs that break functionality, not style nits.

Priority 1 — Missing YAML frontmatter in llm-as-judge-skills agents (4 files)

Your CONTRIBUTING.md requires YAML frontmatter with name and description fields in every skill/agent file. All four files in examples/llm-as-judge-skills/agents/ are missing this block entirely, making them invisible to Claude Code's skill loader.

File Impact
agents/evaluator-agent/evaluator-agent.md Agent cannot be registered or discovered
agents/orchestrator-agent/orchestrator-agent.md Orchestrator cannot be auto-activated
agents/research-agent/research-agent.md Research agent cannot be auto-activated
agents/index.md Index undiscoverable by frontmatter-dependent toolchain

Fix PR: #80


Security Improvements Found (Medium/Low only)

Priority 2 — GitHub token embedded in git clone URL (Medium)

skills/hosted-agents/scripts/sandbox_manager.py line 186 constructs a git clone URL with the token directly in the string:

f"git clone https://x-access-token:{token}@github.com/{repo_url} /workspace"

Even though this file is documented as pseudocode, this pattern teaches an unsafe practice: the token appears in ps aux, shell history, and any command logging infrastructure.

Fix PR: #81

Priority 3 — Unpinned production dependency ranges (Low)

examples/llm-as-judge-skills/package.json uses caret (^) ranges for all 5 production deps, allowing automatic minor version upgrades and raising supply-chain risk.

Fix PR: #82

Priority 4 — Unsanitized user path input in install.sh (Low)

examples/digital-brain-skill/scripts/install.sh line 37 passes the user-supplied custom path directly to mkdir -p and cp -r without validation, allowing .. traversal.

Fix PR: #83


Quality Notes (informational, no PRs)

The following are quality observations only — no action required:

  • examples/digital-brain-skill/agents/AGENTS.md: No declared model, no agent I/O examples, no output format spec. Still scored 70/100.
  • examples/interleaved-thinking/generated_skills/comprehensive-research-agent/SKILL.md: Formatting bug (*Silent should be **Silent in several bullet items) and missing version field in frontmatter.
  • examples/llm-as-judge-skills/agents/orchestrator-agent/orchestrator-agent.md: Lists writer and analyst agents in Available Agents, but no agent directories exist for them. Minor documentation inconsistency.
  • skills/hosted-agents/SKILL.md: Strong guidelines and gotchas, but no ## Examples section with concrete I/O pairs.
  • skills/latent-briefing/SKILL.md: Only one example scenario; a second would improve clarity.

Thank you for a well-organized and useful collection. The core skills are exemplary — several scored 95/100. These fixes should get the overall score above 90.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions