-
-
Notifications
You must be signed in to change notification settings - Fork 657
Remove model name from core-reviewer #120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Opus 4.5 is now out. The pricing for it has dropped dramatically. By raw token counts it's still more expensive than Sonnet 4.5, but Opus 4.5 uses far fewer tokens to do the same work: https://www.anthropic.com/news/claude-opus-4-5#:~:text=New%20on%20the%20Claude%20Developer%20Platform ``` Set to a medium effort level, Opus 4.5 matches Sonnet 4.5’s best score on SWE-bench Verified, but uses 76% fewer output tokens. At its highest effort level, Opus 4.5 exceeds Sonnet 4.5 performance by 4.3 percentage points—while using 48% fewer tokens. ``` As a comparison, this is the cost of running a detailed code review on sonnet: ``` /cost ⎿ Total cost: $1.88 Total duration (API): 10m 30s Total duration (wall): 5m 16s Total code changes: 0 lines added, 0 lines removed Usage by model: claude-3-5-haiku: 40.5k input, 4.0k output, 0 cache read, 0 cache write ($0.0484) claude-opus-4-5: 3.0k input, 7.0k output, 320.0k cache read, 53.0k cache write ($0.68) claude-sonnet: 682 input, 17.5k output, 1.4m cache read, 123.6k cache write ($1.15) ``` And this is Opus: ``` /cost ⎿ Total cost: $0.81 Total duration (API): 3m 9s Total duration (wall): 6m 39s Total code changes: 0 lines added, 0 lines removed Usage by model: claude-3-5-haiku: 11.8k input, 982 output, 0 cache read, 0 cache write ($0.0133) claude-opus-4-5: 1.1k input, 11.7k output, 375.1k cache read, 49.1k cache write ($0.79) ``` The Opus review is of higher quality and consistently finds more medium to high priority issues to resolve.
WalkthroughRemoved the public/front-matter field Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~2 minutes
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
💤 Files with no reviewable changes (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
I think this should be parameterized into a |
Could be done.. I'm new to skills though :-) Any hints or examples on how to do that correctly? |
|
On doing some more research, I think the choices come down to:
I couldn't get Claude to consistently apply the right model if I specified it in CLAUDE.md. If we choose (1), users will have to fork and maintain a local copy if they want changes. Not great, but probably happening already anyway. @obra thoughts? |
|
I vote for option 2 |
|
I'm inclined to remove the explicit model being set. That'd mean we get a sane default and that instructions in CLAUDE.md should allow customization. |
Remove the explicit mention of a model to allow users to customise it as they need.
|
Done! |
Opus 4.5 is now out. The pricing for it has dropped dramatically. By raw token counts it's still more expensive than Sonnet 4.5, but Opus 4.5 uses far fewer tokens to do the same work:
https://www.anthropic.com/news/claude-opus-4-5#:~:text=New%20on%20the%20Claude%20Developer%20Platform
As a comparison, this is the cost of running a detailed code review on sonnet:
And this is Opus:
The Opus review is of higher quality and consistently finds more medium to high priority issues to resolve.
Motivation and Context
How Has This Been Tested?
Breaking Changes
Types of changes
Checklist
Additional context
Summary by CodeRabbit
Documentation
Chores
✏️ Tip: You can customize this high-level summary in your review settings.