docs: add operations guide and config updates #665
Conversation
Summary of ChangesHello @jroth1111, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the operational robustness and configurability of the proxy by introducing detailed documentation for its advanced features and updating the example configuration to reflect these capabilities. The changes aim to provide users with better control over security, performance, and reliability in production environments, covering aspects from secret management and encrypted storage to intelligent routing and comprehensive monitoring. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces extensive documentation for new operational features, including a detailed operations guide, updates to the README, and expanded example configurations. The new documentation is well-structured and comprehensive. My review includes one suggestion to improve the clarity of the routing configuration documentation in docs/operations.md. Consolidating fragmented examples into a single block will help prevent user confusion and align the documentation with the example provided in config.example.yaml.
docs/operations.md
Outdated
| ```yaml | ||
| routing: | ||
| strategy: "fill-first" # default; or "round-robin", "random" | ||
| ``` | ||
|
|
||
| Additional strategies: | ||
| - `least-busy` (uses in-flight request counts) | ||
| - `lowest-latency` (requires `health-tracking.enable: true`) | ||
|
|
||
| ### Fill-first spillover (recommended for “many creds”) | ||
|
|
||
| `fill-first` intentionally “burns” one account first (to stagger rolling-window subscription caps), but with many concurrent terminals it can also overload a single credential, leading to avoidable `429` errors. These knobs keep the intent while adding safe spillover: | ||
|
|
||
| ```yaml | ||
| routing: | ||
| strategy: "fill-first" | ||
| fill-first-max-inflight-per-auth: 4 # 0 = unlimited | ||
| fill-first-spillover: "next-auth" # next-auth (default), least-busy | ||
| ``` | ||
|
|
||
| - When the preferred credential is at capacity (`max-inflight`), selection spills over to another credential instead of overloading one. | ||
| - `next-auth` preserves deterministic “drain first”; `least-busy` maximizes throughput under bursty load. | ||
|
|
||
| Health-aware filtering (requires `health-tracking.enable: true`): | ||
|
|
||
| ```yaml | ||
| routing: | ||
| health-aware: true | ||
| prefer-healthy: true | ||
| ``` |
There was a problem hiding this comment.
The documentation for the routing configuration is split across three separate YAML examples. This fragmentation can be confusing for users, as it might suggest that routing should be declared multiple times, which is invalid in YAML. To improve clarity and prevent misconfiguration, it's better to consolidate all related options into a single, comprehensive example. This helps users understand how different routing settings interact with each other.
Here is a suggested combined example:
routing:
strategy: "fill-first" # fill-first (default), round-robin, random, least-busy, lowest-latency
health-aware: true # Filter unhealthy credentials (COOLDOWN, ERROR)
prefer-healthy: true # Prefer HEALTHY over DEGRADED when health-aware
# Specific to "fill-first" strategy
fill-first-max-inflight-per-auth: 4 # Default: 4. 0 = unlimited
fill-first-spillover: "next-auth" # next-auth (default), least-busy
|
Please keep configuration changes for different PRs separate rather than merging them into a single PR. |
|
Rebased on upstream/main; removed config.example.yaml changes to keep config updates separate and kept docs-only changes (routing example consolidated). @luispater would you mind re-reviewing? |
|
Cleaned up docs-only scope: consolidated routing examples into a single block in docs/operations.md and rebased onto current main so config.example.yaml stays untouched. Ready for re-review. @luispater |
2 participants
Summary
Motivation
Testing