[Enhancement] Add local response caching for repeated LLM queries in GenAI module

[shubhreet11]

Problem:
Every question sent via the GenAI module hits the Sugar-AI
server even if the same question was asked recently. This
wastes quota and slows response time for children.

Proposed Solution

• Add a simple local dict-based or JSON cache in the GenAI folder
• Cache responses for identical questions per session
• Add a cache size limit to avoid memory issues

Expected Outcome
Faster responses for repeated questions + reduced server load.

I'm a GSoC 2026 contributor and would love to work on this!

[shubhreet11]

@mebinthattil Let me know, when can I start working.

[charanrvs09]

Hello, I am a GSoC 2026 applicant and have been contributing to Speak AI (PRs #71, #73, #75, #67 and others related to TTS stability and error handling).

I would like to work on implementing local response caching for repeated LLM queries. My plan is:

• Add a local cache (dictionary/JSON) to store question–response pairs
• Check cache before making API calls
• Add a cache size limit to prevent excessive memory usage
• Optionally persist cache across sessions
• Add logging for cache hits and misses

Please let me know if this approach looks good or if you would like me to follow a different design.

Thank you!

[charanrvs09]

Hello, I have implemented local response caching for repeated LLM queries and opened a Pull Request for this issue.

The implementation includes:

• JSON-based local cache for question–response pairs
• Cache size limit to prevent excessive memory usage
• Persistent cache across sessions
• Normalized cache keys to avoid duplicate entries
• Logging for cache hits, cache misses, and cache size

PR link: #94

Please review and let me know if any changes are needed. Thank you!

[aditii23]

Hii, @mebinthattil and @chimosky
I’m interested in working on this issue.
I have reviewed the codebase and have a good understanding of what needs to be done. I’m confident I can implement a proper solution.
Could you please assign this issue to me? I’ll start working on it right away.
Looking forward to contributing!!

[mebinthattil]

Don't create an issue if you know you're going to be working on it. Create an issue only if you don't intend to work on it. This creates a lot of unnecessary noise for us.

Also you don't need permission to work on something. If there is a contribution you want to make you're free to raise a PR and a maintainer would look into it once they're available.

Issues are not going to be assigned, you can work on it, raise a PR and reference this issue in the PR.

Check the contributing guidelines if you haven't already.