Add an IChatClient implementation to OnnxRuntimeGenAI

[stephentoub]

This adds an IChatClient implementation to ONNX Runtime GenAI, enabling ONNX Runtime GenAI to be used anywhere an IChatClient is being consumed, e.g.

IChatClient chatClient = new OnnxRuntimeGenAIChatClient(...);
await foreach (var update in chatClient.GetStreamingResponseAsync("What is 1 + 2?"))
{
    Console.Write(update);
}

Typically the IChatClient will be configured in a dependency injection container, such that the application code consuming the client needn't be aware of the actual implementation, and the app's setup code can switch between implementations based on ONNX Runtime GenAI, OpenAI, Ollama, etc.

IServiceProvider services = ...;
IChatClient chatClient = services.GetRequiredService<IChatClient>();

[azchohfi]

Then only issue I found is that when ChatOptions is null, it fails for Phi 3.5. This works fine for Phi3.
Phi 3.5 requires max_length to be provided. Seems to be a bug on Phi3.5, but I do not see a way around it, other than always providing max_length.

[natke]

@stephentoub can you please merge main into your branch? This should enable the CIs to pass

[stephentoub]

@stephentoub can you please merge main into your branch? This should enable the CIs to pass

Thanks, but it's already based off of the latest commit in main. It seems like whatever feed is being used may not contain some of the necessary packages, like System.Text.Json?

[pranavsharma]

Can you add a test?

[azchohfi]

@pranavsharma Seems like the package needs to be added to the feed, from the upstream (nuget.org).

[stephentoub]

Can you add a test?

Yes

[azchohfi]

Build is still failing due to the package not being available on the ORT-Nightly feed.
It needs to be manually added. @pranavsharma I do not have access to do that.

[stephentoub]

@pranavsharma, I will update the PR with tests, but I first need your help in getting the feed updated with the necessary dependencies. Otherwise, nothing here even builds.

[baijumeswani]

I can help with that. Is System.Text.Json the only package needed. Will try to have it resolved soon.

[stephentoub]

I can help with that. Is System.Text.Json the only package needed. Will try to have it resolved soon.

Thanks. I'm not sure what's currently available in the feed, but at a minimum you'd need to add Microsoft.Extensions.AI.Abstractions. That itself has a System.Text.Json dependency downlevel.

[stephentoub]

@baijumeswani, also #970 needs Microsoft.ML.Tokenizers

[baijumeswani]

Thanks for the info. A few of our engineering systems team members are out of office this week. And I am trying to find out how to upload nuget packages to our feed. It might take until next week to have this setup.

[baijumeswani]

I have (hopefully) added all the dependencies. Please see the latest windows cpu build.

[luisquintanilla]

Hi folks,

Just checking back in on this. Let us know how we can help. Thanks!

cc: @baijumeswani

[azchohfi]

@baijumeswani The package version got updated, so we need the new packages from upstream saved into ort's feed again. Thanks in advance!

[luisquintanilla]

@baijumeswani is this okay to merge?

[luisquintanilla]

@baijumeswani is this okay to merge?