Comparing Models

Comparing Models

Test the same prompt across multiple models to see how they differ.

Basic Usage

ai.Compare("Explain quantum entanglement in one paragraph").
    On(ai.ModelGPT5, ai.ModelClaudeOpus, ai.ModelGemini3Flash)

Output:

═══════════════════════════════════════════════════════════════
⚡ Comparing 3 models...
═══════════════════════════════════════════════════════════════

▸ openai/gpt-5.2
─────────────────────────────────────────────────────────────
[Response from GPT-5]
(234 tokens)

▸ anthropic/claude-opus-4.5
─────────────────────────────────────────────────────────────
[Response from Claude]
(189 tokens)

▸ google/gemini-3-flash-preview
─────────────────────────────────────────────────────────────
[Response from Gemini]
(156 tokens)

═══════════════════════════════════════════════════════════════

With System Prompt

ai.Compare("Analyze the current market").
    System("You are a financial analyst. Be concise.").
    On(ai.ModelGPT5, ai.ModelClaudeOpus)

With System File

ai.Compare("What do you see?").
    SystemFile("prompts/analyst.md").
    On(ai.ModelGPT5, ai.ModelClaudeOpus, ai.ModelGemini3Flash)

With Variables

ai.Compare("Market update").
    SystemFile("prompts/analyst.md").
    With(ai.Vars{"domain": "crypto"}).
    On(ai.ModelGPT5, ai.ModelClaudeOpus)

Preset Comparisons

Quick shortcuts for common model groups:

// Top flagship models
ai.Compare("Your prompt").TopModels()
// GPT-5, Claude Opus, Gemini 3 Flash, Grok 3

// Fast/cheap models
ai.Compare("Your prompt").FastModels()
// GPT-4o-mini, Claude Haiku, Gemini 2 Flash, Grok 3 Mini

// All GPT variants
ai.Compare("Your prompt").AllGPT()
// GPT-5, GPT-4o, O1

// All Claude variants
ai.Compare("Your prompt").AllClaude()
// Claude Opus, Claude Sonnet, Claude Haiku

Getting Results Programmatically

results := ai.Compare("Hello").
    On(ai.ModelGPT5, ai.ModelClaudeOpus)

for _, r := range results {
    fmt.Printf("Model: %s\n", r.Model)
    fmt.Printf("Response: %s\n", r.Response)
    fmt.Printf("Tokens: %d\n", r.Tokens)
    if r.Error != nil {
        fmt.Printf("Error: %v\n", r.Error)
    }
}

Result Structure

type CompareResult struct {
    Model    Model   // Which model
    Response string  // The response text
    Error    error   // Any error that occurred
    Tokens   int     // Token count
}

Parallel Execution

All models are queried in parallel (using goroutines), so comparison is fast regardless of how many models you test.

Tips

1. Start with TopModels() - See the range of flagship responses
2. Use FastModels() for iteration - Cheaper during development
3. Same system prompt - Ensures fair comparison
4. Check token counts - Some models are more verbose