[BUG] Azure OpenAI deployment not working

[tsoernes]

Environment

GitHub v2-dev branch

Description

I cannot connect to a GPT 4o deployment on Azure using either big-AGI 1.0 or 2.0

This python code works:

import os
from azure.ai.inference import ChatCompletionsClient
from azure.core.credentials import AzureKeyCredential
from dotenv import load_dotenv
from azure.ai.inference.models import SystemMessage, UserMessage

load_dotenv()
api_key = os.getenv("AZURE_INFERENCE_CREDENTIAL", '')
if not api_key:
  raise Exception("A key should be provided to invoke the endpoint")

messages=[
    SystemMessage(content="You are a helpful assistant."),
    UserMessage(content="How many languages are in the world?"),
]

def print_response(response):
    print("Response:", response.choices[0].message.content)
    print("Model:", response.model)
    print("Usage:")
    print("\tPrompt tokens:", response.usage.prompt_tokens)
    print("\tTotal tokens:", response.usage.total_tokens)
    print("\tCompletion tokens:", response.usage.completion_tokens)

client = ChatCompletionsClient(
    endpoint="https://XXXXX.openai.azure.com/models",
    credential=AzureKeyCredential(api_key),
    model="gpt-4o",
)
response = client.complete(messages=messages)
print_response(response)

I have tried using https://XXXx.openai.azure.com/ as URL in big-AGI but it yields:
[OpenAI/azure issue]: Internal Server Error - Service has failed unexpectedly, retrying may help. RequestId:63f8dd8d-150c-494f-91b9-2f45ea0c3a81

Device and browser

Brave

Screenshots and more

No response

Willingness to Contribute

• 🙋‍♂️ Yes, I would like to contribute a fix.

[enricoros]

@tsoernes can you give me an install/credentials to quickly debug this? DM on discord for access details.

[tsoernes]

@enricoros Done. Sent on discord. With both openAI (gpt-4o-mini) and DeepSeek R1 deployments

[enricoros]

@tsoernes thank you, this will help greatly.

[enricoros]

Root cause: while Azure OpenAI services are working correctly, Azure AI Foundry services are not.

AI Foundry does not SEEM to have the /openai/deployments?api-version=2023-03-15-preview method, which we use to auto-
detect models given an endpoint.

PRs welcome to fully support AI Foundry: may require a different model enumeration (API version, path, etc.) or manual definition of models.

Notice added to the app:

[tsoernes]

@enricoros
The full target URL for OpenAI is:
https://MY_DEPLOYMENT.openai.azure.com/openai/deployments/gpt-4o-mini/chat/completions?api-version=2024-08-01-preview

For Foundry it is:
https://MY_DEPLOYMENT2.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview

[tsoernes]

@enricoros Please note: It is not possible to deploy OpenAI services anymore. Everything on Azure has migrated to Foundry.