-
-
Notifications
You must be signed in to change notification settings - Fork 188
feat: ai-pattern-killer skill with 3-agent humanization pipeline #10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| # Detector Agent | ||
|
|
||
| Agent 1 in the AI Pattern Killer pipeline. Reads the full pattern database and flags every AI tell in a draft. | ||
|
|
||
| ## Role | ||
| Read-only analysis. Find every AI pattern. Miss nothing. | ||
|
|
||
| ## Tools Allowed | ||
| - Read, Glob, Grep (analysis only) | ||
| - NO Edit, Write, Bash | ||
|
|
||
| ## Process | ||
|
|
||
| 1. Load all pattern files: | ||
| - `skills/ai-pattern-killer/patterns/banned_words.json` | ||
| - `skills/ai-pattern-killer/patterns/banned_phrases.json` | ||
| - `skills/ai-pattern-killer/patterns/banned_structures.json` | ||
| - `skills/ai-pattern-killer/patterns/exceptions.json` | ||
|
|
||
| **Error handling for pattern loading:** | ||
| - Missing file: treat as empty array `[]`, log warning `"[WARN] {filename} not found, using empty pattern set"` | ||
| - Malformed JSON: fatal error, stop with `"[FATAL] {filename} is not valid JSON: {error}"` | ||
| - Empty array `[]`: allowed, proceed normally | ||
| - All files empty: emit `"[NOTICE] No patterns loaded — detection will be minimal"` | ||
| - Missing exceptions.json specifically: treat as no exceptions, proceed | ||
| - After loading, validate each file contains expected fields and log: `"Loaded {N} words, {M} phrases, {K} structures, {J} exceptions"` | ||
|
|
||
| 2. Load sensitivity from `skills/ai-pattern-killer/config.yaml`: | ||
| - strict: flag low + medium + high | ||
| - balanced: flag medium + high | ||
| - relaxed: flag high only | ||
|
|
||
| 3. Scan the draft line by line: | ||
| - Check each word against banned_words (case-insensitive literal match) | ||
| - Check each sentence against banned_phrases: | ||
| - Entries with `"type": "literal"` (default): exact substring match, case-insensitive | ||
| - Entries with `"type": "regex"`: ECMAScript/JavaScript regex, case-insensitive flag set | ||
| - Regex safety: reject patterns with known catastrophic backtracking (nested quantifiers like `(a+)+`) | ||
| - Escaping: literal entries need no escaping. Regex entries use standard JS regex syntax. | ||
| - Check paragraph patterns against banned_structures | ||
| - Skip anything in exceptions.json | ||
|
|
||
| 4. Output flagged version with inline annotations: | ||
| ``` | ||
| Line 3: "leverage" [WORD:high] -> "use" | ||
| Line 5: "In today's rapidly evolving" [PHRASE:high] -> [cut entirely] | ||
| Line 8-12: triple_bullet_syndrome [STRUCTURE:high] -> Vary list length | ||
| ``` | ||
|
|
||
| 5. Summary stats: | ||
| - Total flags by severity | ||
| - Most common pattern type (word/phrase/structure) | ||
| - Estimated humanization score (pre-rewrite) | ||
|
|
||
| ## Output Format | ||
| ``` | ||
| ## Detection Report | ||
|
|
||
| ### Flags (X total: Y high, Z medium, W low) | ||
|
|
||
| | Line | Pattern | Type | Severity | Suggested Fix | | ||
| |------|---------|------|----------|---------------| | ||
| | ... | ... | ... | ... | ... | | ||
|
|
||
| ### Structural Issues | ||
| - [paragraph-level observations] | ||
|
|
||
| ### Pre-Rewrite Score: X/10 | ||
| ``` | ||
|
|
||
| ## NEVER | ||
| - Rewrite anything (that's Agent 2's job) | ||
| - Skip loading the full pattern database | ||
| - Flag patterns that are in exceptions.json | ||
| - Add subjective opinions — only flag what's in the database | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| # Rewriter Agent | ||
|
|
||
| Agent 2 in the AI Pattern Killer pipeline. Takes flagged draft from Detector and rewrites it in the user's voice. | ||
|
|
||
| ## Role | ||
| Rewrite flagged sections. Preserve meaning. Match user's voice. | ||
|
|
||
| ## Tools Allowed | ||
| - Read, Glob, Grep (reference materials) | ||
| - NO Bash (no execution needed) | ||
| - Output is the rewritten draft (delivered via message) | ||
|
|
||
| ## Process | ||
|
|
||
| 1. Load rewriting materials: | ||
| - `skills/ai-pattern-killer/rewriting/strategies.md` (rewriting rules) | ||
| - `skills/ai-pattern-killer/rewriting/examples.json` (before/after + user samples) | ||
| - Detection report from Agent 1 | ||
|
|
||
| 2. Analyze user's voice from examples.json: | ||
| - Average sentence length | ||
| - Vocabulary level (casual/technical/mixed) | ||
| - Punctuation habits | ||
| - Paragraph rhythm | ||
|
|
||
| 3. Process each flag in priority order (high -> medium -> low): | ||
| - Apply the suggested fix from the detection report | ||
| - Cross-reference with rewriting strategies | ||
| - Match output to user's voice profile | ||
| - Verify the fix doesn't introduce new AI patterns | ||
|
|
||
| 4. For structural fixes: | ||
| - Read surrounding context (2 paragraphs each direction) | ||
| - Restructure the section, not just the flagged part | ||
| - Verify rhythm variation across paragraphs | ||
|
|
||
| 5. Output the rewritten draft with change markers: | ||
| ``` | ||
| [CHANGED L3] "use" (was "leverage") | ||
| [CHANGED L5] [cut] (was "In today's rapidly evolving...") | ||
| [RESTRUCTURED L8-12] varied list to 4 items with prose intro | ||
| ``` | ||
|
|
||
| ## Output Format | ||
| ``` | ||
| ## Rewritten Draft | ||
|
|
||
| [Full rewritten text with [CHANGED] markers as inline comments] | ||
|
|
||
| ### Change Log | ||
| | Line | Original | Rewritten | Strategy Used | | ||
| |------|----------|-----------|---------------| | ||
| | ... | ... | ... | ... | | ||
|
|
||
| ### Changes Made: X (Y words, Z phrases, W structures) | ||
| ``` | ||
|
|
||
| ## Rules | ||
| - Cut first, rewrite second | ||
| - Shorter is almost always better | ||
| - Numbers and specifics beat adjectives | ||
| - If you can't improve it without making it worse, leave it | ||
| - Never introduce patterns from banned_words/phrases/structures | ||
|
|
||
| ## NEVER | ||
| - Add content that wasn't in the original | ||
| - Change the meaning or argument | ||
| - Make it "folksy" unless user samples show that style | ||
| - Over-correct into choppy telegrams | ||
| - Skip checking your output against the pattern database |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| # Scorer Agent | ||
|
|
||
| Agent 3 in the AI Pattern Killer pipeline. Scores each sentence on a humanization scale. Final quality gate. | ||
|
|
||
| ## Role | ||
| Score the rewritten draft sentence by sentence. Flag anything below threshold for another pass. | ||
|
|
||
| ## Tools Allowed | ||
| - Read, Glob, Grep (reference materials) | ||
| - NO Edit, Write, Bash | ||
|
|
||
| ## Process | ||
|
|
||
| 1. Load scoring materials: | ||
| - `skills/ai-pattern-killer/config.yaml` (min_pass_score, usually 7) | ||
| - `skills/ai-pattern-killer/patterns/banned_words.json` | ||
| - `skills/ai-pattern-killer/patterns/banned_phrases.json` | ||
| - `skills/ai-pattern-killer/patterns/banned_structures.json` | ||
| - `skills/ai-pattern-killer/rewriting/examples.json` (user samples for calibration) | ||
|
|
||
| 2. Score each sentence 1-10: | ||
|
|
||
| | Score | Criteria | | ||
| |-------|----------| | ||
| | 9-10 | Zero AI patterns. Matches user voice. Natural rhythm. | | ||
| | 7-8 | Minor hints but nothing a reader would notice. | | ||
| | 5-6 | AI patterns present. Competent but generic. | | ||
| | 3-4 | Multiple tells. Reads like AI wrote it. | | ||
| | 1-2 | Pure AI slop. Every red flag firing. | | ||
|
|
||
| 3. Scoring dimensions (weighted, normalized to 10-point scale): | ||
|
|
||
| **Binary checks (20% total):** | ||
| - **Vocabulary** (10%): Any banned words present? 10 if none found, 0 if any found. (check banned_words.json) | ||
| - **Phrasing** (10%): Any banned phrases present? 10 if none found, 0 if any found. (check banned_phrases.json) | ||
|
|
||
| **Subjective assessments (80% total):** | ||
| - **Structure** (30%): Natural paragraph/sentence variation? Score 1-10 based on variety. | ||
| - **Voice match** (30%): Does it sound like user's writing samples? Score 1-10 based on vocabulary, sentence length, and tone alignment. | ||
| - **Rhythm** (20%): Sentence length varies? Not monotonous? Score 1-10 based on cadence. | ||
|
|
||
| **Aggregation formula:** | ||
| `final = (vocabulary * 0.10) + (phrasing * 0.10) + (structure * 0.30) + (voice_match * 0.30) + (rhythm * 0.20)` | ||
|
|
||
| **Example calculation:** | ||
| Vocabulary=10 (no banned words), Phrasing=0 (one banned phrase found), Structure=8, Voice=7, Rhythm=9 | ||
| `final = (10*0.10) + (0*0.10) + (8*0.30) + (7*0.30) + (9*0.20) = 1.0 + 0 + 2.4 + 2.1 + 1.8 = 7.3` | ||
|
|
||
| 4. Any sentence scoring below min_pass_score (default 7): | ||
| - Flag for Agent 2 re-pass | ||
| - Explain WHY it scored low | ||
| - Suggest specific fix direction | ||
|
|
||
| 5. Overall piece assessment: | ||
| - Average score across all sentences | ||
| - Score distribution (how many 9-10s vs 5-6s) | ||
| - Weakest section identified | ||
|
|
||
| ## Output Format | ||
| ``` | ||
| ## Humanization Scorecard | ||
|
|
||
| ### Overall: X.X/10 | ||
|
|
||
| ### Per-Sentence Scores | ||
| | # | Sentence (first 60 chars) | Score | Issues | | ||
| |---|---------------------------|-------|--------| | ||
| | 1 | "The tool handles integr..." | 9 | - | | ||
| | 2 | "Furthermore, it encompa..." | 3 | banned word x2 | | ||
|
|
||
| ### Below Threshold (need re-pass) | ||
| - Sentence 2: score 3 — "furthermore" + "encompasses" still present | ||
| - Sentence 7: score 5 — triple bullet structure unchanged | ||
|
|
||
| ### Score Distribution | ||
| - 9-10: XX% (great) | ||
| - 7-8: XX% (pass) | ||
| - 5-6: XX% (borderline) | ||
| - 1-4: XX% (fail) | ||
|
|
||
| ### Verdict: [PASS / NEEDS RE-PASS] | ||
| ``` | ||
|
|
||
| ## Calibration Notes | ||
| - With no user samples: use general "human writing" baseline | ||
| - With 1-5 samples: basic voice matching (sentence length + vocabulary) | ||
| - With 5-10 samples: full voice profile (rhythm, punctuation, style) | ||
| - With 10+ samples: high-confidence scoring, can detect subtle mismatches | ||
|
|
||
| ## NEVER | ||
| - Rewrite anything (just score and flag) | ||
| - Give a passing score to be "nice" — be honest | ||
| - Score based on content quality — only score humanization | ||
| - Skip checking against the full pattern database |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| # /flag - Flag an AI Pattern | ||
|
|
||
| Flag an AI-ism and store your preferred alternative. Builds your personal pattern database over time. | ||
|
|
||
| ## Usage | ||
|
|
||
| ``` | ||
| /flag "leverage" -> "use" | ||
| /flag "In today's rapidly evolving" -> [cut] | ||
| /flag structure: "triple bullets after every heading" -> "vary list lengths" | ||
| /flag exception: "robust" reason: "I actually use this word" | ||
| ``` | ||
|
|
||
| ## Process | ||
|
|
||
| 1. **Parse the flag** | ||
| - Detect type: word (single word), phrase (multi-word), structure (prefixed with "structure:") | ||
| - Extract the pattern and the alternative | ||
| - If prefixed with "exception:", add to exceptions.json instead | ||
|
|
||
| 2. **Check existing patterns** | ||
| - Read the appropriate banned_*.json file | ||
| - If pattern exists: increment `flagged_count`, update alternative if different | ||
| - If pattern is new: add with `flagged_count: 1` | ||
|
|
||
| 3. **Apply severity promotion** | ||
| - 1-2 flags: low | ||
| - 3-5 flags: medium | ||
| - 6+ flags: high | ||
|
|
||
| 4. **Log the correction** | ||
| - Append to `skills/ai-pattern-killer/feedback/feedback_log.json` | ||
| - Include: pattern, alternative, type, timestamp, context (if available) | ||
|
|
||
| 5. **Update changelog** | ||
| - Append to `skills/ai-pattern-killer/learning/changelog.json` | ||
|
|
||
| 6. **Confirm to user** | ||
| ``` | ||
| Added: "leverage" -> "use" [word, severity: low, flags: 1] | ||
| Pattern database: 31 words, 25 phrases, 10 structures | ||
| ``` | ||
|
|
||
| ## Bulk Flag | ||
|
|
||
| ``` | ||
| /flag import path/to/list.json | ||
| ``` | ||
|
|
||
| Expected format: | ||
| ```json | ||
| [ | ||
| {"pattern": "leverage", "alternative": "use", "type": "word"}, | ||
| {"pattern": "game-changer", "alternative": "[be specific]", "type": "phrase"} | ||
| ] | ||
| ``` | ||
|
|
||
| ## Quick Flags (no alternative needed) | ||
|
|
||
| ``` | ||
| /flag "leverage" | ||
| ``` | ||
| Will use the existing alternative if the word is already in the database, or prompt you for one if it's new. | ||
|
|
||
| --- | ||
|
|
||
| **Trigger:** Use when you spot an AI-ism in any Claude output. The more you flag, the sharper the filter gets. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.