Feature Request: Support for Multiple Model Backends (e.g., via LiteLLM Interface)

[NewGenesis04]

Hi team 👋

First off, thank you for the amazing work on this project! It’s has a lot of potential.

I'd love to suggest a feature that could make the system more flexible: support for multiple model backends. It would be awesome if users could easily switch between different LLM providers (e.g., OpenAI, Mistral, Anthropic, Cohere, etc.) without major code changes.

One way to enable this could be by integrating an abstraction layer like LiteLLM, which offers a unified interface for multiple models. This would:

• Give users more deployment options
• Improve portability
• Allow for fallback or multimodel routing

Let me know if this is something you're open to, and I’d be happy to help brainstorm or contribute if needed!

Thanks again 🙏

[samtiz]

Hi @NewGenesis04,

Thanks for the great suggestion!

We are also considering an abstraction layer like LiteLLM or OpenRouter, but we're not quite sure which to choose or how to best implement it.

If you have a moment, could you take a look at our latest code in src/common/ai and see if you think LiteLLM would be a good fit? If the compatibility looks good, we're very open to integrating it.

[PhantomInTheWire]

@samtiz I think this should use the Vercel SDK instead of LiteLLM, since afaik LiteLLM doesn’t have a native JS/TS SDK. Let me know if you agree, happy to ship a PR for this.