Skip to content

Commit 7ec19ae

Browse files
feat: improve AI reviewer assumption handling to reduce spam (#13)
* feat: improve AI reviewer assumption handling to reduce spam Update the prompt to prevent the AI from spamming users with questions about unverifiable issues. Changes: - Default to skipping issues that can't be verified from the diff - Only ask for clarification on critical issues (security, breaking bugs, data loss) - Require explicit labeling of inferences as "Inference (not verified)" - Emphasize focusing on directly visible issues in the diff This makes reviews more honest and actionable while avoiding noise from assumptions about code not shown in the diff. * debug: temporarily enable debug output to diagnose empty response * debug: add API response logging when content is empty * feat: add structured review format with verdict and overall feedback - Require short overall feedback summary at the start (1-2 sentences) - Enforce one of three verdicts at the end: Approved, Approved with recommendations, or Request changes - Remove debug logging and restore DEBUG_MODE conditional - Ensure every review has clear approval status
1 parent 508d037 commit 7ec19ae

File tree

1 file changed

+13
-1
lines changed

1 file changed

+13
-1
lines changed

ai-reviewer.sh

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -218,7 +218,18 @@ PROMPT="You are an expert code reviewer. Please analyze this code diff and provi
218218
219219
Focus on security, performance, code quality, and best practices.
220220
221-
Focus on high-value issues. Style suggestions are welcome if impactful, but not minor optimizations. Be concise and dense - use bullet points for clear structure. Avoid repetition - in summary sections, only repeat critical issues (security, bugs, breaking changes). If flagging issues about code not visible in the diff, clearly state what you're assuming and why. For non-critical improvements, consider approving with recommendations rather than requesting changes.
221+
Focus on high-value issues. Style suggestions are welcome if impactful, but not minor optimizations. Be concise and dense - use bullet points for clear structure. Avoid repetition - in summary sections, only repeat critical issues (security, bugs, breaking changes). Important: Focus on issues directly visible in the diff. If you cannot verify something from the diff alone (e.g., missing context, unclear defaults, code not shown):
222+
- Default: Skip the issue to avoid spam
223+
- Only ask for clarification if it's critical (security vulnerabilities, breaking bugs, data loss risks): \"Cannot verify [X] from diff - please confirm [specific question]\"
224+
- If making an inference about non-critical issues, explicitly label it: \"Inference (not verified): [observation]\"
225+
226+
Review Structure:
227+
1. Start with a short overall feedback summary (1-2 sentences)
228+
2. Then provide detailed findings
229+
3. End with one of these verdicts ONLY:
230+
- \"✅ Approved\" (no issues found)
231+
- \"✅ Approved with recommendations\" (minor improvements suggested, but not blocking)
232+
- \"❌ Request changes\" (critical issues that must be fixed before merge)
222233
223234
Required JSON format:
224235
{
@@ -231,6 +242,7 @@ Instructions:
231242
1. Respond with a single valid JSON object
232243
2. Include the Friendly AI Reviewer footer with heart emoji at the end of the review field
233244
3. For labels_added, prefer existing repository labels when possible
245+
4. Always end your review with one of the three verdict options listed above before the footer
234246
235247
Code to review:
236248
$PROMPT_PREFIX

0 commit comments

Comments
 (0)