How to add cloudflare workers AI via litellm proxy UI?

[tan-yong-sheng]

Since cloudflare has officially supported OpenAI-compatible api for Cloudflare workers AI (see https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/), I'm thinking how we could we add it into llmlite proxy UI?

For example, the sample API_BASE_URL will be "https://api.cloudflare.com/client/v4/accounts/${env.CLOUDFLARE_ACCOUNT_ID}/ai/v1"

Here is the UI that I am tinkering with to add in this Cloudflare Workers AI....

Hope for help, thanks.

[tan-yong-sheng]

I put these on config.yaml instead

model_list:
  - model_name: cloudflare/llama-3-8b-instruct
    litellm_params:
      model: cloudflare/@cf/meta/llama-3-8b-instruct

  - model_name: cloudflare/llama-2-7b-chat-fp16
    litellm_params:
      model: cloudflare/@cf/meta/llama-2-7b-chat-fp16

  - model_name: cloudflare/mistral-7b-instruct-v0.1
    litellm_params:
      model: cloudflare/@cf/mistral/mistral-7b-instruct-v0.1

  - model_name: cloudflare/deepseek-coder-6.7b-base-awq
    litellm_params:
      model: cloudflare/@hf/thebloke/deepseek-coder-6.7b-instruct-awq