fix: Add conversational context to interactive CLI chat #465

nithish-95 · 2025-11-30T22:19:57Z

Fixes #351
This PR addresses an issue where the interactive chat in the CLI was not sending the full conversational context with each message. This resulted in the model not being able to remember previous parts of the conversation.

The following changes have been made:

A messages slice is now maintained to store the history of the conversation.
On each new user message, the entire conversation history is sent to the model.
The assistant's response is then appended to the history for the next turn.

This ensures that the model has the full context of the conversation, as demonstrated in the updated behavior.

gemini-code-assist · 2025-11-30T22:20:13Z

Summary of Changes

Hello @nithish-95, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a critical issue where the interactive chat in the command-line interface lacked conversational memory. By implementing a robust mechanism to store and transmit the full exchange of messages, the system can now provide a more natural and coherent chat experience, ensuring the underlying model retains context throughout the conversation.

Highlights

Conversational Context for Interactive CLI Chat: The interactive chat functionality in the CLI now maintains conversational context, allowing the model to remember previous messages and responses within a single session.
Conversation History Management: A messages slice is introduced to store the full history of the conversation. Each new user message is appended to this history, and the entire history is sent with subsequent requests to the model. The assistant's response is also appended to this history.
Improved JSON Handling for NIM Chat: The chatWithNIM function now uses proper JSON marshaling for request payloads and unmarshaling for streamed responses, replacing basic string parsing for content extraction.
New Test for Context Preservation: A new test file, nim_chat_test.go, has been added to specifically verify that conversational context is correctly maintained across multiple turns in the NIM chat.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

sourcery-ai

Hey there - I've reviewed your changes - here's some feedback:

The new Message type in nim.go is very similar in shape to desktop.OpenAIChatMessage; consider reusing or sharing a common chat message struct to avoid duplication and keep the OpenAI-compatible payload types consistent across the codebase.
Relying on the mutable package-level nimDefaultPort (especially modified in tests) makes the behavior more implicit; it may be more robust to inject the base URL/port into chatWithNIM (or into a small NIM client struct) so tests and callers can configure endpoints without altering global state.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The new `Message` type in `nim.go` is very similar in shape to `desktop.OpenAIChatMessage`; consider reusing or sharing a common chat message struct to avoid duplication and keep the OpenAI-compatible payload types consistent across the codebase.
- Relying on the mutable package-level `nimDefaultPort` (especially modified in tests) makes the behavior more implicit; it may be more robust to inject the base URL/port into `chatWithNIM` (or into a small NIM client struct) so tests and callers can configure endpoints without altering global state.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

gemini-code-assist

Code Review

This pull request successfully adds conversational context to the interactive CLI chat for both NIM and other models. It correctly maintains a history of messages and sends the full context with each request. The changes also include significant improvements, such as replacing manual JSON string construction and parsing with standard library functions (json.Marshal and json.Unmarshal), making the code more robust and secure. A new test has been added to verify the context-aware behavior for NIM chat, which is excellent. My review includes a few suggestions to improve code clarity, reduce redundancy, and fix a slightly misleading comment.

cmd/cli/commands/nim.go

cmd/cli/desktop/desktop.go

…nagement in chat functions

…add package declaration to nim.go.

ilopezluna

Thanks for working on this @nithish-95 !
I have tried your PR and it works well, nice 👏
Added a couple of minor suggestions, and a questions related to llamacppgo and download.go changes (I think we should not change these files).
Let me know what do you think 🙏

ilopezluna · 2025-12-01T09:54:23Z

pkg/inference/backends/llamacpp/download.go

@@ -28,7 +27,7 @@ const (
 var (
 	ShouldUseGPUVariant       bool
 	ShouldUseGPUVariantLock   sync.Mutex
-	ShouldUpdateServer        = true
+	ShouldUpdateServer        = false


Why should we not update llama.cpp?

Would be nice to have an explanation of why this needed to be changed alright. Maybe the PR is not ready for review?

ilopezluna · 2025-12-01T09:56:54Z

pkg/inference/backends/llamacpp/llamacpp.go

@@ -95,8 +95,8 @@ func (l *llamaCpp) Install(ctx context.Context, httpClient *http.Client) error {

 	// We don't currently support this backend on Windows. We'll likely
 	// never support it on Intel Macs.
-	if (runtime.GOOS == "darwin" && runtime.GOARCH == "amd64") ||
-		(runtime.GOOS == "windows" && runtime.GOARCH != "amd64" && runtime.GOARCH != "arm64") {
+	// We don't currently support this backend on Windows.


Suggested change

// We don't currently support this backend on Windows.

Yes duplicate

ilopezluna · 2025-12-01T09:58:19Z

pkg/inference/backends/llamacpp/llamacpp.go

-	if (runtime.GOOS == "darwin" && runtime.GOARCH == "amd64") ||
-		(runtime.GOOS == "windows" && runtime.GOARCH != "amd64" && runtime.GOARCH != "arm64") {
+	// We don't currently support this backend on Windows.
+	if (runtime.GOOS == "windows" && !(runtime.GOARCH == "amd64" || runtime.GOARCH == "arm64")) {


Is it Intel Mac supported? If so we should update the comment. If not we should get back the if (runtime.GOOS == "darwin" && runtime.GOARCH == "amd64") ||

Personally I'd vote for removing lines 96 to 101 completely. But this is fine.

When somebody encounters an error from a new platform, I think this is good. It would be nice to encourage people to explore these errors and even enable new platforms (from a community perspective).

I think we try too hard to block code execution for things deemed unsupported. If the community wants to work on Intel Mac and get that working, so be it.

Now whether than propagates to Docker Desktop, is a different discussion (and unlikely since it's a legacy platform). We have the same discussion for Linux DMR.

ilopezluna · 2025-12-01T10:02:41Z

cmd/cli/commands/nim.go

+	requestPayload := struct {
+		Model    string                      `json:"model"`
+		Messages []desktop.OpenAIChatMessage `json:"messages"`
+		Stream   bool                        `json:"stream"`
+	}{
+		Model:    modelName,
+		Messages: *messages,
+		Stream:   true,
+	}


Suggested change

requestPayload := struct {

Model string `json:"model"`

Messages []desktop.OpenAIChatMessage `json:"messages"`

Stream bool `json:"stream"`

}{

Model: modelName,

Messages: *messages,

Stream: true,

}

requestPayload := desktop.OpenAIChatRequest{

Model: modelName,

Messages: *messages,

Stream: true,

}

ericcurtin

Could the NIM client and client for other backends be consolidated further? @nithish-95 poke us again when it is ready for review. Please test thoroughly (although I have faith you already are). Thanks for pushing this

ericcurtin · 2025-12-02T10:47:47Z

I meant to press comment instead of approve :)

feat: introduce conversation history management for chat commands

011d5e3

sourcery-ai bot reviewed Nov 30, 2025

View reviewed changes

gemini-code-assist bot reviewed Nov 30, 2025

View reviewed changes

cmd/cli/commands/nim.go Outdated Show resolved Hide resolved

cmd/cli/desktop/desktop.go Outdated Show resolved Hide resolved

cmd/cli/desktop/desktop.go Show resolved Hide resolved

nithish-95 added 3 commits November 30, 2025 14:53

refactor: unify chat message type and improve conversation history ma…

a642cbb

…nagement in chat functions

refactor: Update chat message types to desktop.OpenAIChatMessage

e13f02c

feat: Enable Intel Mac support, default server updates to false, and …

4448de4

…add package declaration to nim.go.

ilopezluna reviewed Dec 1, 2025

View reviewed changes

ericcurtin approved these changes Dec 2, 2025

View reviewed changes

fix: Add conversational context to interactive CLI chat #465

Are you sure you want to change the base?

fix: Add conversational context to interactive CLI chat #465

Conversation

nithish-95 commented Nov 30, 2025

Uh oh!

gemini-code-assist bot commented Nov 30, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ilopezluna left a comment

Choose a reason for hiding this comment

Uh oh!

ilopezluna Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

ericcurtin Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

ilopezluna Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

ericcurtin Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

ilopezluna Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

ericcurtin Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ilopezluna Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

ericcurtin left a comment

Choose a reason for hiding this comment

Uh oh!

ericcurtin commented Dec 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ericcurtin Dec 2, 2025 •

edited

Loading