docs: comprehensive README update with LLM config and advanced features (#1707)

maxprilutskiy · web-flow · commit b2d335b37af3 · 2025-12-23T15:49:28.000-08:00
- Added complete LLM provider configuration guide (OpenAI, Anthropic,
  Google, Groq, Mistral, OpenRouter, Ollama)
- Added environment variables and API key setup for all providers
- Added locale-pair model mapping with wildcard patterns
- Added custom translation prompts documentation
- Added manual translation overrides (data-lingo-override) with examples
- Added build modes documentation (translate vs cache-only)
- Added custom locale resolvers documentation
- Added configuration options table with defaults
- Added development, locale persistence, and pluralization configuration
- Updated feature list with all capabilities
- Fixed demo app paths in examples
diff --git a/.changeset/comprehensive-readme-update.md b/.changeset/comprehensive-readme-update.md
@@ -0,0 +1,20 @@
+---
+"@lingo.dev/compiler": patch
+---
+
+docs: comprehensive README update with LLM configuration, manual overrides, and advanced features
+
+Added extensive documentation covering:
+- Complete LLM provider configuration (OpenAI, Anthropic, Google, Groq, Mistral, OpenRouter, Ollama)
+- Environment variables for all supported providers
+- Locale-pair model mapping with wildcard patterns
+- Custom translation prompts
+- Manual translation overrides using data-lingo-override attribute
+- Build modes (translate vs cache-only) and recommended workflows
+- Custom locale resolvers (server and client)
+- Configuration options table with defaults
+- Development configuration (pseudotranslator, translation server port)
+- Locale persistence configuration (cookie settings)
+- Pluralization configuration
+- Updated feature list
+- Fixed demo app paths in examples
diff --git a/packages/new-compiler/README.md b/packages/new-compiler/README.md
@@ -13,7 +13,10 @@ automatically transforms React components to inject translation calls.
 - **Opt-in or automatic** - Configure whether to require `'use i18n'` directive or transform all files
 - **Multi-bundler support** - Works with Vite, Webpack and Next.js (both Webpack and Turbopack builds)
 - **Translation server** - On-demand translation generation during development
-- **AI-powered translations** - Support for multiple LLM providers and Lingo.dev Engine
+- **AI-powered translations** - Support for multiple LLM providers (OpenAI, Anthropic, Google Gemini, Groq, Mistral, OpenRouter, Ollama) and Lingo.dev Engine
+- **Manual overrides** - Override AI translations for specific locales using `data-lingo-override` attribute
+- **Custom locale resolvers** - Provide your own locale detection and persistence logic
+- **Automatic pluralization** - Detects and converts messages to ICU MessageFormat
 
 ## Getting started
 
@@ -58,7 +61,7 @@ Install the package - `pnpm install @lingo.dev/compiler`
   }
   ```
 
-See `demo/vite-react-spa` for the working example
+See `demo/new-compiler-vite-react-spa` for the working example
 
 ### Next.js
 
@@ -99,6 +102,274 @@ See `demo/vite-react-spa` for the working example
   }
   ```
 
+See `demo/new-compiler-next16` for the working example
+
+## Configuration Options
+
+### Core Configuration
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `sourceRoot` | `string` | `"src"` | Root directory of the source code |
+| `lingoDir` | `string` | `"lingo"` | Directory for lingo files (`.lingo/`) |
+| `sourceLocale` | `LocaleCode` | **(required)** | Source locale (e.g., `"en"`, `"en-US"`) |
+| `targetLocales` | `LocaleCode[]` | **(required)** | Target locales to translate to |
+| `useDirective` | `boolean` | `false` | Whether to require `'use i18n'` directive |
+| `models` | `string \| Record<string, string>` | `"lingo.dev"` | Model configuration (see below) |
+| `prompt` | `string` | `undefined` | Custom translation prompt |
+| `buildMode` | `"translate" \| "cache-only"` | `"translate"` | Build mode (see below) |
+
+### Development Configuration
+
+Configure development-specific behavior via the `dev` option:
+
+```ts
+{
+  dev: {
+    // Use pseudotranslator (fake translations) instead of real AI
+    usePseudotranslator: boolean;  // default: false
+
+    // Starting port for translation server
+    translationServerStartPort: number;  // default: 60000
+
+    // Custom translation server URL (advanced)
+    translationServerUrl?: string;
+  }
+}
+```
+
+### Locale Persistence
+
+Configure how locale changes are persisted:
+
+```ts
+{
+  localePersistence: {
+    type: "cookie",
+    config: {
+      name: string;    // default: "locale"
+      maxAge: number;  // default: 31536000 (1 year)
+    }
+  }
+}
+```
+
+### Pluralization
+
+Configure automatic pluralization detection:
+
+```ts
+{
+  pluralization: {
+    enabled: boolean;  // default: true
+    model: string;     // default: "groq:llama-3.1-8b-instant"
+  }
+}
+```
+
+## Using LLMs for Translation
+
+### Lingo.dev Engine (Recommended)
+
+The simplest way to get started is using Lingo.dev Engine:
+
+```ts
+{
+  models: "lingo.dev"
+}
+```
+
+Set your API key in `.env`:
+
+```bash
+LINGODOTDEV_API_KEY=your_api_key_here
+```
+
+Get your API key at [lingo.dev](https://lingo.dev)
+
+### Direct LLM Providers
+
+You can use any supported LLM provider directly. Configure using locale-pair mapping:
+
+```ts
+{
+  models: {
+    // Specific locale pair
+    "en:es": "google:gemini-2.0-flash",
+
+    // Wildcard patterns
+    "*:de": "groq:llama3-8b-8192",         // All translations to German
+    "en:*": "openai:gpt-4o",               // All translations from English
+    "*:*": "anthropic:claude-3-5-sonnet"   // Fallback for all other pairs
+  }
+}
+```
+
+**Supported Providers:**
+
+| Provider | Model String Format | Environment Variable | Get API Key |
+|----------|---------------------|---------------------|-------------|
+| **OpenAI** | `openai:gpt-4o` | `OPENAI_API_KEY` | [platform.openai.com](https://platform.openai.com/account/api-keys) |
+| **Anthropic** | `anthropic:claude-3-5-sonnet` | `ANTHROPIC_API_KEY` | [console.anthropic.com](https://console.anthropic.com/get-api-key) |
+| **Google** | `google:gemini-2.0-flash` | `GOOGLE_API_KEY` | [ai.google.dev](https://ai.google.dev/) |
+| **Groq** | `groq:llama3-8b-8192` | `GROQ_API_KEY` | [groq.com](https://groq.com) |
+| **Mistral** | `mistral:mistral-large` | `MISTRAL_API_KEY` | [console.mistral.ai](https://console.mistral.ai) |
+| **OpenRouter** | `openrouter:anthropic/claude-3.5-sonnet` | `OPENROUTER_API_KEY` | [openrouter.ai](https://openrouter.ai) |
+| **Ollama** | `ollama:llama2` | *(none required)* | [ollama.com/download](https://ollama.com/download) |
+
+**Example with multiple providers:**
+
+```ts
+{
+  sourceLocale: "en",
+  targetLocales: ["es", "de", "fr", "ja"],
+  models: {
+    "en:es": "groq:llama3-8b-8192",        // Fast & cheap for Spanish
+    "en:de": "google:gemini-2.0-flash",    // Good for German
+    "*:*": "openai:gpt-4o"                 // High quality for everything else
+  }
+}
+```
+
+### Custom Translation Prompts
+
+Customize the translation prompt using placeholders:
+
+```ts
+{
+  models: "lingo.dev",
+  prompt: "Translate from {SOURCE_LOCALE} to {TARGET_LOCALE}. Use a formal tone and preserve all technical terms."
+}
+```
+
+## Manual Translation Overrides
+
+Override AI-generated translations for specific text using the `data-lingo-override` attribute:
+
+```tsx
+export function Welcome() {
+  return (
+    <div>
+      {/* Override translations for brand name */}
+      <h1 data-lingo-override={{ de: "Lingo.dev", fr: "Lingo.dev", es: "Lingo.dev" }}>
+        Lingo.dev
+      </h1>
+
+      {/* Override only specific locales */}
+      <p data-lingo-override={{ de: "Professionelle Übersetzung" }}>
+        Professional translation
+      </p>
+
+      {/* Works with rich text and interpolations */}
+      <p data-lingo-override={{
+        de: "Willkommen <strong0>{name}</strong0>",
+        fr: "Bienvenue <strong0>{name}</strong0>"
+      }}>
+        Welcome <strong>{name}</strong>
+      </p>
+    </div>
+  );
+}
+```
+
+The `data-lingo-override` attribute:
+- Accepts an object with locale codes as keys
+- Supports partial overrides (only specify locales you want to override)
+- Is automatically removed from the final output
+- Works with locale region codes (e.g., `en-US`, `en-GB`)
+- Supports rich text with component placeholders (e.g., `<strong0>`)
+
+**Use cases:**
+- Brand names that shouldn't be translated
+- Technical terms requiring specific translations
+- Legal text requiring certified translations
+- Marketing copy that needs human review
+
+## Build Modes
+
+Control how translations are handled at build time:
+
+### `translate` (default)
+
+Generate missing translations at build time using configured LLM:
+
+```ts
+{
+  buildMode: "translate"
+}
+```
+
+- Generates translations for any missing entries
+- Fails build if translation fails
+- Best for: Development and CI pipelines with API keys
+
+### `cache-only`
+
+Only use existing cached translations:
+
+```ts
+{
+  buildMode: "cache-only"
+}
+```
+
+- No API calls made during build
+- Fails build if translations are missing
+- Best for: Production builds without API keys
+- Requires translations to be pre-generated (during dev or in CI)
+
+**Environment Variable Override:**
+
+```bash
+LINGO_BUILD_MODE=cache-only npm run build
+```
+
+**Recommended Workflow:**
+
+1. **Development**: Use `translate` mode with `usePseudotranslator: true`
+2. **CI**: Generate real translations with `buildMode: "translate"` and real API keys
+3. **Production Build**: Use `buildMode: "cache-only"` (no API keys needed)
+
+## Custom Locale Resolvers
+
+Customize how locales are detected and persisted by providing custom resolver files:
+
+### Custom Server Locale Resolver
+
+Create `.lingo/locale-resolver.server.ts` (Next.js):
+
+```ts
+// Custom server-side locale detection
+export async function getServerLocale(): Promise<string> {
+  // Your custom logic - e.g., from database, headers, etc.
+  const { headers } = await import('next/headers');
+  const headersList = await headers();
+  const acceptLanguage = headersList.get('accept-language');
+
+  // Parse and return locale
+  return acceptLanguage?.split(',')[0]?.split('-')[0] || 'en';
+}
+```
+
+### Custom Client Locale Resolver
+
+Create `.lingo/locale-resolver.client.ts`:
+
+```ts
+// Custom client-side locale detection and persistence
+export function getClientLocale(): string {
+  // Your custom logic - e.g., from localStorage, URL params, etc.
+  return localStorage.getItem('user-locale') || 'en';
+}
+
+export function persistLocale(locale: string): void {
+  localStorage.setItem('user-locale', locale);
+  // Optionally update URL, etc.
+}
+```
+
+If these files don't exist, the compiler uses the default cookie-based implementation.
+
 ## Development
 
 `pnpm install` from project root
