Add draft for harnessing DeepSeek-R1 with .NET Aspire and Ollama locally

source/_drafts/harnessing-deepseek-r1-with-dotnet-aspire-and-ollama-locally.md

@@ -0,0 +1,223 @@
+---
+title: 'Harnessing DeepSeek-R1 with .NET Aspire and Ollama locally'
+permalink: /2025/01/28/harnessing-deepseek-r1-with-dotnet-aspire-and-ollama-locally/
+date: 1/28/2025 9:11:14 AM
+disqusIdentifier: 20250128091114
+coverSize: partial
+tags: [AI, C#, LLM, Ollama, LLM, SLM]
+coverCaption: 'Liebvillers, France'
+coverImage: 'https://live.staticflickr.com/65535/54288775875_2a5a3ffb62_h.jpg'
+thumbnailImage: 'https://live.staticflickr.com/65535/54288775875_cf354c8df3_q.jpg'
+---
+In my earlier posts, I've demonstrated how to install Ollama using the Windows installer. However, for developers, there's a more streamlined method to set up Ollama on your machine.
+
+In this blog post, we'll explore how to run DeepSeek-R1 by harnessing the capabilities of .NET Aspire alongside Ollama on your local environment.
+<!-- more -->
+# Introduction
+Ollama offers a Docker image that can be seamlessly executed on your local machine. By integrating Docker with .NET Aspire, you create a robust solution for running diverse large language models (LLMs) and small language models (SLMs) efficiently. And you still can use your beefy GPU to accelerate the answer generation process.
+
+This setup ensures that Ollama operates within an isolated containerized environment, providing enhanced security and consistency without the necessity of installing additional software directly on your machine.
+
+# Prerequisites
+
+Before proceeding, ensure that you have the following prerequisites installed on your machine:
+<p></p>
+
+- [.NET 9 SDK](https://dotnet.microsoft.com/download)
+- [Docker Desktop](https://www.docker.com/products/docker-desktop)
+
+# Setting up DeepSeek-R1 with .NET Aspire and Ollama
+
+## Step 1: Create a new .NET Aspire project
+
+To run DeepSeek-R1 with .NET Aspire and Ollama on your machine, we will start by creating a new .NET Aspire project. Open a terminal window and run the following command to create a new .NET Aspire project:
+
+```bash
+mkdir DeepSeekOllamaAspire\
+cd .\DeepSeekOllamaAspire\
+dotnet new aspire-starter
+cd .\DeepSeekOllamaAspire.AppHost\
+dotnet add package CommunityToolkit.Aspire.Hosting.Ollama  // 👈🏼 .NET Aspire Ollama Hosting
+cd ..\DeepSeekOllamaAspire.ApiService\
+dotnet add package CommunityToolkit.Aspire.OllamaSharp  // 👈🏼 .NET Aspire Ollama Client
+```
+
+## Step 2: Configure .NET Aspire AppHost
+
+The code for the .NET Aspire AppHost is configuring the Ollama server hosting integration. It is adding the Ollama container to the app host, configuring data volume, GPU support, and container lifetime. Additionally, it adds the DeepSeek-R1 model to the Ollama server.
+
+```csharp
+var builder = DistributedApplication.CreateBuilder(args);
+
+// 👇🏼 Configure Ollama server hosting integration
+var ollama =
+    builder.AddOllama("ollama") // 👈🏼 Add Ollama container to the app host
+        .WithDataVolume() // 👈🏼 Adds a data volume to store models
+        .WithGPUSupport() // 👈🏼 Use your beefy GPU 🎉
+        .WithLifetime(ContainerLifetime.Persistent); // 👈🏼 Keep the container running
+                                                     //    when the app exits
+
+// 👇🏼 Add DeepSeek-R1 model to Ollama server
+var chat =
+    ollama.AddModel("chat", "deepseek-r1:1.5b");
+
+var apiService = 
+    builder.AddProject<Projects.DeepSeekOllamaAspire_ApiService>("apiservice")
+           // 👇🏼 Our Web API relies on Ollama chat and waits for it to start 
+           .WithReference(chat)
+           .WaitFor(chat);
+
+builder.AddProject<Projects.DeepSeekOllamaAspire_Web>("webfrontend")
+    .WithExternalHttpEndpoints()
+    .WithReference(apiService)
+    .WaitFor(apiService);
+
+builder.Build().Run();
+```
+
+## Step 3: Configure .NET Aspire ApiService
+
+The code for the .NET Aspire ApiService is configuring the Ollama client integration. It is adding the Ollama client to the IOC and registering it as a chat client. Then it is used to call the Ollama API to get an answer from DeepSeek-R1.
+
+```csharp
+using Microsoft.Extensions.AI;
+
+var builder = WebApplication.CreateBuilder(args);
+
+// Add service defaults & Aspire client integrations.
+builder.AddServiceDefaults();
+
+// 👇🏼 IOC registration of OllamaSharp client as IChatClient
+builder.AddOllamaSharpChatClient("chat");
+
+builder.Services.AddProblemDetails();
+builder.Services.AddOpenApi();
+
+var app = builder.Build();
+
+app.UseExceptionHandler();
+
+if (app.Environment.IsDevelopment())
+{
+    app.MapOpenApi();
+}
+
+// 👇🏼 Web API getting IChatClient from IOC
+app.MapGet("/chat", async (IChatClient chatClient, string question) =>
+{
+    // 👇🏼 Calling Ollama API to get an answer from DeepSeek-R1
+    var response = await chatClient.CompleteAsync(question);
+    return new Response(response.Message.Text ?? "Failed to generate response.");
+})
+.WithName("Chat");
+
+app.MapDefaultEndpoints();
+
+app.Run();
+
+public record Response(string Value);
+```
+
+## Step 4: The web frontend
+
+The web frontend is a simple Blazor application. It sends the question to the API service and displays the answer. It doesn't know anything about Ollama or DeepSeek-R1. It just knows about the API service.
+
+We create a ChatApiClient that uses HttpClient to call the API service. The ChatAsync method sends the question to the API service and returns the answer.
+
+```csharp
+public class ChatApiClient(HttpClient httpClient)
+{
+    public async Task<string> ChatAsync(string question, 
+                                        CancellationToken cancellationToken = default)
+    {
+        var requestUri = $"/chat?question={question}";
+        var response = 
+            await httpClient.GetFromJsonAsync<Response>(requestUri, cancellationToken);
+        return response?.Value ?? "";
+    }
+}
+
+public record Response(string Value);
+```
+
+The `ChatApiClient` is registered in the `Program.cs` file.
+
+```csharp
+builder.Services.AddHttpClient<ChatApiClient>(client =>
+    {
+        // This URL uses "https+http://" to indicate HTTPS is preferred over HTTP.
+        // Learn more about service discovery scheme resolution at https://aka.ms/dotnet/sdschemes.
+        client.BaseAddress = new("https+http://apiservice");
+    });
+```
+
+and injected and used in the `Chat.razor` component
+
+```csharp
+@page "/chat"
+@attribute [StreamRendering(true)]
+@attribute [OutputCache(Duration = 5)]
+
+@inject ChatApiClient ChatApi
+
+<PageTitle>Chat</PageTitle>
+
+<h1>Chat</h1>
+
+<p>This component demonstrates showing data loaded from a 
+backend API service talking to Ollama.</p>
+
+@if (_response == null)
+{
+    <p><em>Loading...</em></p>
+}
+else
+{
+    <p>@_response</p>
+}
+
+@code {
+    private string? _response;
+
+    protected override async Task OnInitializedAsync()
+    {
+        var question = Uri.EscapeDataString("Hello, can you help me in C#?");
+        _response = await ChatApi.ChatAsync(question);
+    }
+}
+```
+
+# Running the .NET Aspire project
+
+To run the .NET Aspire project, execute the following command in the terminal:
+
+```bash
+cd .\DeepSeekOllamaAspire.AppHost\
+dotnet run
+```
+
+The .NET Aspire project will start, and you will see the Container "ollama" starting. The OllamaModelResource "chat" will download the DeepSeek-R1 model. When the Ollama server is ready, the ApiService will start, and the Web frontend will be available.
+
+![.NET Aspire Ollama DeepSeek-R1](/images/aspire-ollama-deepseek-r1.png)
+
+Click on the Project "webfrontend" link and then on the "Chat" link to see the answer generated by DeepSeek-R1.
+
+![.NET Aspire Ollama DeepSeek-R1 answer](/images/aspire-ollama-deepseek-r1-answer.png)
+
+And with the power of .NET Aspire, you can observe the trace of the calls to the Ollama API.
+
+![.NET Aspire Ollama DeepSeek-R1 trace](/images/aspire-ollama-deepseek-r1-trace.png)
+
+# Conclusion
+By integrating DeepSeek-R1 with .NET Aspire and Ollama, developers can leverage a robust and efficient setup for managing and deploying language models. This solution provides an easy streamlined approach to running LLM or SLM on your local machine without the need for additional software installations.
+
+# References
+
+* [.NET Aspire Community Toolkit Ollama integration](https://learn.microsoft.com/en-us/dotnet/aspire/community-toolkit/ollama?tabs=dotnet-cli%2Cdocker)
+* [Ollama Docker image](https://hub.docker.com/r/ollama/ollama)
+* [Ollama DeepSeek-R1 model](https://www.ollama.com/library/deepseek-r1)
+* [Using Local AI models with .NET Aspire](https://devblogs.microsoft.com/dotnet/local-ai-models-with-dotnet-aspire/)
+
+Get the source code on GitHub [laurentkempe/aiPlayground/DeepSeekOllamaAspire](https://github.com/laurentkempe/aiPlayground/tree/main/DeepSeekOllamaAspire)
+<p></p>
+{% githubCard user:laurentkempe repo:aiPlayground align:left %}