Skip to content
This repository was archived by the owner on May 25, 2026. It is now read-only.
This repository was archived by the owner on May 25, 2026. It is now read-only.

feat: Add request caching for repeated queries #40

Closed

Description

Summary

Implement optional caching for AI responses to reduce API calls and improve response times for repeated queries.

Use Cases

  1. Development/Testing - Same prompts during debugging
  2. Deterministic outputs - When temperature=0 and same input
  3. Cost savings - Avoid redundant API calls

Proposed Implementation

1. Cache configuration

// ~/.config/opencode/puter.json
{
  "cache_enabled": true,
  "cache_ttl_ms": 3600000,
  "cache_max_entries": 100,
  "cache_directory": "~/.cache/opencode/puter-responses"
}

2. Cache key generation

function generateCacheKey(request: ChatRequest): string {
  return hash({
    model: request.model,
    messages: request.messages,
    temperature: request.temperature,
    // ... other deterministic params
  });
}

3. Cache invalidation

  • TTL-based expiration
  • Manual cache clear command
  • Model-specific cache policies

Technical Notes

  • Use file-based cache for persistence
  • Consider memory cache for hot entries
  • Skip caching for streaming responses (or cache final result)

Priority: Low

Activity

  1. chindris-mihai-alexandru commented on Feb 15, 2026

    @chindris-mihai-alexandru
    OwnerAuthor

    Consolidated into #48 to keep roadmap tracking in one place.

  2. chindris-mihai-alexandru commented on Feb 15, 2026

    @chindris-mihai-alexandru
    OwnerAuthor

    Reopening to track this item as a standalone issue while also keeping #48 as the roadmap index.

  3. chindris-mihai-alexandru commented on Mar 13, 2026

    @chindris-mihai-alexandru
    OwnerAuthor

    Implemented file-backed response caching (config options, cache TTL/max, cache clear CLI). See commit fbb2428.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestroadmapTracked in roadmap/meta issuesstatus:backlogPlanned but not in progress

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions