Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
0ebef87
moonshotai/kimi-k2-thinking
LearningCircuit Nov 9, 2025
bce90c5
Update ai-reviewer.sh
LearningCircuit Nov 9, 2025
247071c
feat: add comprehensive debug logging for thinking model responses
LearningCircuit Nov 9, 2025
1b7a5e9
fix: parse JSON from thinking model responses and add heart icon
LearningCircuit Nov 9, 2025
21f0815
fix: simplify thinking model response handling
LearningCircuit Nov 9, 2025
23c534c
fix: escape forward slash in sed command
LearningCircuit Nov 9, 2025
d0593da
debug: enable debug mode to see full AI response
LearningCircuit Nov 9, 2025
c568830
fix: remove markdown code blocks from AI response
LearningCircuit Nov 9, 2025
13e7589
debug: remove content limits to see full AI response
LearningCircuit Nov 9, 2025
18175ee
fix: resolve bash syntax error and enable markdown code block removal
LearningCircuit Nov 9, 2025
7b60b31
debug: output raw AI response to logs instead of file
LearningCircuit Nov 9, 2025
9b5bb86
fix: properly handle thinking model responses with multiline regex
LearningCircuit Nov 9, 2025
195e886
fix: correct YAML indentation for Cleanup step
LearningCircuit Nov 9, 2025
e2f3d22
fix: increase AI_MAX_TOKENS to 8000 to prevent response truncation
LearningCircuit Nov 9, 2025
156c4d3
fix: remove max_tokens limit to prevent response truncation
LearningCircuit Nov 9, 2025
940934a
refactor: address AI review recommendations
LearningCircuit Nov 9, 2025
86fe9f0
style: remove emoji from header, keep heart in footer only
LearningCircuit Nov 9, 2025
8e41e39
style: use 'Friendly AI Reviewer' instead of 'F-AI-R' in footer
LearningCircuit Nov 9, 2025
7b58b0e
fix: use constants consistently for review header and footer
LearningCircuit Nov 9, 2025
efacc8f
fix: increase AI_MAX_TOKENS to 64000 to prevent response truncation
LearningCircuit Nov 9, 2025
68777f9
feat: include human comments and limit AI review context
LearningCircuit Nov 9, 2025
8fa95c0
feat: include PR description and commit messages in review context
LearningCircuit Nov 9, 2025
40df378
docs: update README with accurate costs and new features
LearningCircuit Nov 9, 2025
66f6cf6
fix: respect DEBUG_MODE flag for all debug output
LearningCircuit Nov 9, 2025
ebe3641
feat: improve AI review focus and conciseness
LearningCircuit Nov 9, 2025
3872041
refine: reduce repetition in AI reviews and clarify assumptions
LearningCircuit Nov 9, 2025
3465b23
refactor: simplify review structure and clarify assumptions
LearningCircuit Nov 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 26 additions & 7 deletions .github/workflows/ai-code-reviewer.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ on:

jobs:
comprehensive-review:
name: 🤖 AI Code Review
name: AI Code Review
runs-on: ubuntu-latest
if: github.event.action == 'labeled' && github.event.label.name == 'ai_code_review'
permissions:
Expand Down Expand Up @@ -34,9 +34,9 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
AI_MODEL: ${{ vars.AI_MODEL || 'z-ai/glm-4.6' }}
AI_MODEL: ${{ vars.AI_MODEL || 'moonshotai/kimi-k2-thinking' }}
AI_TEMPERATURE: ${{ vars.AI_TEMPERATURE || '0.1' }}
AI_MAX_TOKENS: ${{ vars.AI_MAX_TOKENS || '2000' }}
AI_MAX_TOKENS: ${{ vars.AI_MAX_TOKENS || '64000' }}
MAX_DIFF_SIZE: ${{ vars.MAX_DIFF_SIZE || '800000' }}
EXCLUDE_FILE_PATTERNS: ${{ vars.EXCLUDE_FILE_PATTERNS || '*.lock,*.min.js,*.min.css,package-lock.json,yarn.lock' }}
PR_NUMBER: ${{ github.event.pull_request.number }}
Expand All @@ -48,10 +48,11 @@ jobs:
echo "Running AI code review..."
AI_RESPONSE=$(cat diff.txt | bash ai-reviewer.sh)

# Save response for debugging (only in debug mode)
# Output raw response for debugging (only in debug mode)
if [ "$DEBUG_MODE" = "true" ]; then
echo "Debug: AI response saved to ai_response.txt"
echo "$AI_RESPONSE" > ai_response.txt
echo "=== RAW AI RESPONSE FOR DEBUG ==="
echo "$AI_RESPONSE"
echo "=== END RAW AI RESPONSE ==="
fi

# Check if AI_RESPONSE is empty
Expand All @@ -63,8 +64,26 @@ jobs:
# Parse JSON response
if ! echo "$AI_RESPONSE" | jq . >/dev/null 2>&1; then
echo "⚠️ AI response is not valid JSON. Cannot process review."

# Log raw response for debugging (redact sensitive info)
echo "=== DEBUG: Raw AI Response (first 2000 chars) ==="
echo "$AI_RESPONSE" | head -c 2000
echo ""
echo "=== END DEBUG ==="

# Try to extract JSON from thinking model response
if echo "$AI_RESPONSE" | grep -q '"content"'; then
echo "=== DEBUG: Attempting to extract content from response ==="
EXTRACTED_CONTENT=$(echo "$AI_RESPONSE" | jq -r '.choices[0].message.content // empty' 2>/dev/null || echo "")
if [ -n "$EXTRACTED_CONTENT" ]; then
echo "Extracted content (first 1000 chars):"
echo "$EXTRACTED_CONTENT" | head -c 1000
echo ""
fi
fi

# Don't post raw response as it may contain sensitive data
echo "AI response could not be processed. Please check the workflow logs." | gh pr comment ${{ github.event.pull_request.number }} --repo ${{ github.repository }} -F -
echo "AI response could not be processed. Please check the workflow logs for debugging information." | gh pr comment ${{ github.event.pull_request.number }} --repo ${{ github.repository }} -F -
exit 1
fi

Expand Down
72 changes: 56 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,21 @@
# Friendly AI Reviewer (FAIR) Setup Guide
# Friendly AI Reviewer Setup Guide

- Creates highly-customizable AI Reviews as PR comments.
- Installation: Just 2 files copied to your repo and a open router API Key in your secrets.
- Costs: $0.01 - $0.05 per review (depends highly on model)
- ** Example output ** https://github.com/LearningCircuit/local-deep-research/pull/959#issuecomment-3445396169
- Installation: Just 2 files copied to your repo and an OpenRouter API Key in your secrets.
- Costs: $0.01 - $0.05 per review (even for large PRs with full context)
- **Example output**: https://github.com/LearningCircuit/local-deep-research/pull/959#issuecomment-3445396169

This guide explains how to set up the automated AI PR review system using OpenRouter to analyze pull requests with your choice of AI model.

## What's New

**Latest Updates:**
- **Thinking Model Support**: Now supports advanced reasoning models like Kimi K2 that use `<thinking>` tags
- **Rich Context**: Includes PR descriptions, commit messages, and human comments for comprehensive reviews
- **Higher Token Limits**: Default 64k tokens for complete reviews without truncation
- **Smart Context Management**: Only fetches most recent AI review to save tokens
- **Enhanced Error Handling**: Robust parsing of various AI response formats

## Overview

The AI Code Reviewer provides automated, comprehensive code reviews covering:
Expand Down Expand Up @@ -45,12 +54,20 @@ The review is posted as a single comprehensive comment on your pull request with

### 3. Configure Workflow (Optional)

The workflow is pre-configured with sensible defaults, but you can customize it by editing `.github/workflows/ai-code-reviewer.yml`:
The workflow is pre-configured with sensible defaults, but you can customize it by setting repository variables in **Settings** → **Secrets and variables** → **Actions** → **Variables**:

- **AI_MODEL**: Change the AI model (see [OpenRouter models](https://openrouter.ai/models))
- **AI_MODEL**: Change the AI model (default: `moonshotai/kimi-k2-thinking`)
- See [OpenRouter models](https://openrouter.ai/models) for options
- Recommended: Models with reasoning capabilities (Kimi K2, o1, etc.)
- **AI_TEMPERATURE**: Adjust randomness (default: `0.1` for consistent reviews)
- **AI_MAX_TOKENS**: Maximum response length (default: `2000`)
- **AI_MAX_TOKENS**: Maximum response length (default: `64000`)
- High limit ensures comprehensive reviews without truncation
- For large PRs with thinking models, this prevents cut-off responses
- Adjust lower for cost savings on smaller PRs
- **MAX_DIFF_SIZE**: Maximum diff size in bytes (default: `800000` / 800KB)
- **DEBUG_MODE**: Enable debug logging (default: `false`)
- ⚠️ Warning: Exposes code diff in workflow logs when enabled
- Only enable temporarily for troubleshooting

## Usage

Expand Down Expand Up @@ -79,11 +96,31 @@ The AI posts a comprehensive comment analyzing your code across all focus areas.

## Cost Estimation

Costs vary by model, but most code-focused models on OpenRouter are very affordable:
- Typical small PR (< 1000 lines): $0.001 - $0.01
- Large PR (1000-5000 lines): $0.01 - $0.05
Costs with the default Kimi K2 thinking model are very affordable. Based on real usage data:

**Typical Costs:**
- Small PR (< 1000 lines): $0.01 - $0.02
- Medium PR (1000-3000 lines): $0.02 - $0.04
- Large PR (3000+ lines): $0.04 - $0.06

**Example from a 20-commit PR with full context:**
- Input: ~5,000-9,000 tokens (diff + PR description + commits + human comments)
- Output: ~2,000-6,000 tokens (comprehensive review)
- **Total cost: $0.01 - $0.05 per review**

**Why So Affordable:**
- Kimi K2 has competitive pricing (~$0.001-$0.003 per 1k tokens)
- Smart context management (only most recent AI review, limited commit history)
- Most PRs are smaller than you think in token count
- The 64k token limit is a ceiling, not typical usage

**Cost varies based on:**
- PR size (larger diffs = more input tokens)
- Review complexity (detailed reviews = more output tokens)
- Number of human comments and commit messages included
- OpenRouter provider routing (prices vary slightly by provider)

Check [OpenRouter pricing](https://openrouter.ai/models) for specific model costs.
Check [OpenRouter pricing](https://openrouter.ai/models) for current Kimi K2 rates.

## Customization

Expand Down Expand Up @@ -138,11 +175,14 @@ If you get a "Diff is too large" error:

The workflow fetches and sends these repository elements to the AI:
1. **Code Changes**: Full diff of modified files
2. **Labels**: All repository labels with descriptions and colors
3. **Previous Comments**: All previous AI review comments from the PR
4. **CI/CD Status**: GitHub Actions check runs and build statuses
5. **PR Metadata**: Pull request details, head SHA, repository information
6. **Files**: May include sensitive configuration files, keys, or credentials
2. **PR Description**: Title and description text from the pull request
3. **Commit Messages**: Up to 15 most recent commit messages (excluding merges)
4. **Human Comments**: All comments from human reviewers on the PR
5. **Labels**: All repository labels with descriptions and colors
6. **Previous AI Review**: Most recent AI review comment only (limited to 10k chars)
7. **CI/CD Status**: GitHub Actions check runs and build statuses
8. **PR Metadata**: Pull request details, head SHA, repository information
9. **Files**: May include sensitive configuration files, keys, or credentials

### Recommended Mitigations

Expand Down
Loading