Test/realtime chat

[iceljc]

PR Type

enhancement

---

Description

• Introduced ChatStreamMiddleware for real-time chat streaming via WebSockets

• Added models for chat stream event deserialization

• Refactored and improved real-time completion providers for OpenAI and Google AI

• Registered new middleware in web application startup

• Minor bug fixes and code cleanups in related files

---

Changes walkthrough 📝

	Relevant files
Enhancement	15 files ChatStreamMiddleware.cs Add middleware for real-time chat streaming via WebSockets +159/-0  ChatStreamEventResponse.cs Add models for chat stream event deserialization                  +15/-0    Program.cs Register new chat streaming middleware in app startup        +4/-1      Using.cs Add global using for chat stream models                                    +2/-1      ChatHubMiddleware.cs Rename middleware class for chat hub                                          +2/-2      RealTimeCompletionProvider.cs Refactor Google AI real-time provider for clarity and maintainability +29/-27  RealTimeCompletionProvider.cs Refactor OpenAI real-time provider and improve session handling +51/-27  RealtimeChatSession.cs Make chat session class internal and update type names      +4/-4      AsyncWebsocketDataCollectionResult.cs Make async websocket data collection result class internal +1/-1      AsyncWebsocketDataResultEnumerator.cs Make async websocket data result enumerator class internal +1/-1      AiWebsocketPipelineResponse.cs Make pipeline response class internal                                        +1/-2      ChatSessionUpdate.cs Rename session update class for clarity                                    +2/-2      RealtimeHub.cs Refactor model connection call for clarity                              +2/-1      IRealTimeCompletion.cs Refactor Connect method signature for readability                +2/-1      Program.cs Increase buffer size for audio streaming                                  +1/-1
Bug fix	1 files RoutingContext.cs Remove unused parameter in agent options retrieval              +1/-1
Configuration changes	1 files CrontabPlugin.cs Comment out hosted service registration for crontab            +2/-2
Formatting	2 files ConversationItemCreated.cs Add newline and formatting for model class                              +1/-0      ConversationController.cs Add newline at end of file                                                              +1/-1

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

[qodo-merge-pro]

Qodo Merge was enabled for this repository. To continue using it, please link your Git account with your Qodo account here.

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Error Handling The error handling in the WebSocket connection is minimal. The middleware catches exceptions but doesn't properly close the WebSocket connection in error scenarios, which could lead to resource leaks. try { var services = httpContext.RequestServices; var segments = request.Path.Value.Split("/"); var agentId = segments[segments.Length - 2]; var conversationId = segments[segments.Length - 1]; using var webSocket = await httpContext.WebSockets.AcceptWebSocketAsync(); await HandleWebSocket(services, agentId, conversationId, webSocket); } catch (Exception ex) { _logger.LogError(ex, $"Error when connecting Chat stream. ({ex.Message})"); } return; } Missing Initialization The OpenAI RealTimeCompletionProvider constructor is missing initialization for the _settings parameter, which could lead to null reference exceptions when the settings are accessed. RealtimeModelSettings settings, ILogger<RealTimeCompletionProvider> logger, IServiceProvider services, BotSharpOptions botsharpOptions) { _settings = settings; _logger = logger; _services = services; _botsharpOptions = botsharpOptions; } Parameter Change The GetAgentOptions method call was modified to remove the 'byName: true' parameter, which might change the behavior of agent lookup by name. var agents = agentService.GetAgentOptions([agentId]).Result;

[qodo-merge-pro]

Qodo Merge was enabled for this repository. To continue using it, please link your Git account with your Qodo account here.

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	✅ Fix WebSocket fragmentation handling Suggestion Impact: The commit completely refactored the WebSocket handling logic, replacing the direct buffer handling with a more sophisticated BotSharpRealtimeSession class that properly handles WebSocket communication including fragmentation. While not implementing the exact suggestion, it addresses the same underlying issue by using a more comprehensive approach. code diff: + _session?.Dispose(); + _session = new BotSharpRealtimeSession(services, webSocket, new ChatSessionOptions + { + BufferSize = 1024 * 16, + JsonOptions = BotSharpOptions.defaultJsonOptions + }); + var hub = services.GetRequiredService<IRealtimeHub>(); var conn = hub.SetHubConnection(conversationId); + conn.CurrentAgentId = agentId; // load conversation and state var convService = services.GetRequiredService<IConversationService>(); convService.SetConversationId(conversationId, []); await convService.GetConversationRecordOrCreateNew(agentId); - var buffer = new byte[1024 * 32]; - WebSocketReceiveResult result; - - do + await foreach (ChatSessionUpdate update in _session.ReceiveUpdatesAsync(CancellationToken.None)) { - result = await webSocket.ReceiveAsync(new(buffer), CancellationToken.None); - - if (result.MessageType != WebSocketMessageType.Text) - { - continue; - } - - var receivedText = Encoding.UTF8.GetString(buffer, 0, result.Count); + var receivedText = update?.RawResponse; if (string.IsNullOrEmpty(receivedText)) { continue; } var (eventType, data) = MapEvents(conn, receivedText); - if (eventType == "start") { await ConnectToModel(hub, webSocket); @@ -92,28 +88,20 @@ else if (eventType == "disconnect") { await hub.Completer.Disconnect(); + break; } } - while (!webSocket.CloseStatus.HasValue); - await webSocket.CloseAsync(result.CloseStatus.Value, result.CloseStatusDescription, CancellationToken.None); + _session?.Disconnect(); + _session?.Dispose(); The code is using a fixed-size buffer for receiving WebSocket messages, but it doesn't handle fragmented messages correctly. WebSocket messages can be split across multiple frames, and the current implementation will process each frame independently, potentially causing data corruption for large messages. src/Plugins/BotSharp.Plugin.ChatHub/ChatStreamMiddleware.cs [61-77] var buffer = new byte[1024 * 32]; WebSocketReceiveResult result; +var messageBuffer = new List<byte>(); do { result = await webSocket.ReceiveAsync(new(buffer), CancellationToken.None); if (result.MessageType != WebSocketMessageType.Text) { continue; } - var receivedText = Encoding.UTF8.GetString(buffer, 0, result.Count); - if (string.IsNullOrEmpty(receivedText)) + messageBuffer.AddRange(new ArraySegment<byte>(buffer, 0, result.Count)); + + if (result.EndOfMessage) { - continue; - } + var receivedText = Encoding.UTF8.GetString(messageBuffer.ToArray()); + messageBuffer.Clear(); + + if (string.IsNullOrEmpty(receivedText)) + { + continue; + } [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 8 __ Why: The suggestion correctly identifies a significant issue with WebSocket message handling. The current implementation doesn't properly handle fragmented messages, which could lead to data corruption for large messages. The improved code properly accumulates message fragments before processing.	Medium
	✅ Check WebSocket state The SendEventToUser method doesn't check if the WebSocket is in a valid state before sending data. If the client has disconnected or the connection is closing, this could throw an exception and crash the middleware. src/Plugins/BotSharp.Plugin.ChatHub/ChatStreamMiddleware.cs [106] -await SendEventToUser(webSocket, data); +if (webSocket.State == WebSocketState.Open) +{ + await SendEventToUser(webSocket, data); +} [Suggestion has been applied] Suggestion importance[1-10]: 7 __ Why: This suggestion addresses an important defensive programming practice by checking the WebSocket state before sending data. Without this check, the application could throw exceptions if the client disconnects unexpectedly, potentially causing middleware crashes.	Medium
	✅ Avoid potential deadlocks The code is using .Result to synchronously wait for an asynchronous operation, which can lead to deadlocks in ASP.NET applications. This is a common anti-pattern that should be avoided. src/Infrastructure/BotSharp.Core/Routing/RoutingContext.cs [85-93] // Convert id to name if (!Guid.TryParse(agentId, out _)) { var agentService = _services.GetRequiredService<IAgentService>(); - var agents = agentService.GetAgentOptions([agentId]).Result; + var agents = agentService.GetAgentOptions([agentId]).GetAwaiter().GetResult(); if (agents.Count > 0) { agentId = agents.First().Id; [Suggestion has been applied] Suggestion importance[1-10]: 5 __ Why: The suggestion identifies a potential issue with using .Result, but the proposed solution of using GetAwaiter().GetResult() is only marginally better. While it avoids some deadlock scenarios, the method should ideally be refactored to be fully async with await.	Low
Update

[qodo-merge-pro]

Suggestion: Check WebSocket state

Suggested change

	await SendEventToUser(webSocket, data);
	if (webSocket.State == WebSocketState.Open)
	{
	await SendEventToUser(webSocket, data);
	}

[Oceania2018]

[GGHansome]

Auto Review Result:

Code Review Summary

Change Summary: This change introduces a new concept of "Links" into the Agent model, alongside enhancements to the template rendering system and vector database interactions. It includes additional methods for managing Links, updates to template registration to support links rendering, and a more robust method for interacting with WebSocket connections.

Identified Issues

Issue 1: Null Reference Handling

• Description: In the SetLinks method and several other parts, when assigning potentially null collections, the code uses [] which is not a valid C# syntax for initializing an empty list or array.
• Suggestion: Replace [] with new List<Type>() to correctly handle the instantiation.
• Example:

  // Before
  Links = links ?? [];
  // After
  Links = links ?? new List<AgentLink>();

Issue 2: Missing Null Check

• Description: In the RenderAgentLinks method, there is no null check of agent.Links before using it. Even though it may be assigned an empty list in several places, a null reference check is a good defensive programming habit.
• Suggestion: Add a null check before accessing properties or methods.
• Example:

  agent.Links ??= new List<AgentLink>(); // Ensures Links is never null

Issue 3: Inconsistent Newline Use

• Description: There is inconsistent newline usage at the end of files, which can cause issues with some source control systems.
• Suggestion: Ensure all files end with a newline character.

Overall Evaluation

The code changes are clear and introduce useful extensions to the existing system. However, attention should be given to proper null handling and initializing of collections to prevent potential runtime errors. Additionally, consistent code formatting, particularly regarding newlines, should be maintained for code health and compatibility with various tools.

[GGHansome]

Auto Review Result:

Code Review Summary

Change Overview: The changes introduce a new feature called Links to the agent model in the BotSharp application, which includes modifications in several classes and interfaces to support the new functionality. The changes also involve updates to handle Link entities in the setup, storage, cloning, and rendering processes of agent information.

Issues Found

Issue 1: [Potential Bug in Default Value Assignment]

• Description: The SetLinks method assigns an empty array [] instead of new List<AgentLink>() if the links parameter is null. This might cause a runtime error as [] is not valid syntax in C# for initializing a list.
• Suggestion: Use new List<AgentLink>() instead of [] to ensure proper initialization.
• Example:

  // Before
  public Agent SetLinks(List<AgentLink> links)
  {
      Links = links ?? [];
      return this;
  }
  // After
  public Agent SetLinks(List<AgentLink> links)
  {
      Links = links ?? new List<AgentLink>();
      return this;
  }

Issue 2: [Missing Null Check]

• Description: The RenderAgentLinks method does not explicitly handle the possibility of agent.Links being null before iterating over it, although it appears to rely on initializing with an empty list.
• Suggestion: Add a null check for agent.Links to make the code more robust.
• Example:

  // Add this line before the iteration
  if (agent.Links == null) return;

Issue 3: [Code Style Consistency]

• Description: The use of return; in methods for early exit is mixed with direct return of values. It’s good practice to be consistent in code style definitions.
• Suggestion: Consider maintaining consistency in return statements, potentially avoiding redundant return; statements in void methods.

Issue 4: [Inconsistent File Names Handling]

• Description: The code contains different methods for obtaining file names, such as using Path.GetFileName() and string manipulation. This can introduce complexities and potential bugs in path handling.
• Suggestion: Utilize Path class methods consistently for better code maintainability and readability.
• Example:

  // Ensure usage of Path methods such as
  var fileName = Path.GetFileName(file);

Overall Evaluation

The code changes are logically structured to introduce a Link feature; however, attention to detail in initialization syntax, null checks, and style consistency would enhance code quality. Ensure to use robust exception handling and utilize appropriate collection initializations in C#. Additionally, harmonizing file handling methods will improve readability and prevent potential errors.

[GGHansome]

Auto Review Result:

Review Summary

Change Overview: The code changes introduce a new feature for handling AgentLink in the system, which is similar to the already existing AgentTemplate structure. Additional methods and properties were added to various parts of the system to support this new Link type. Furthermore, the code enhances the template rendering logic by allowing registration and management of tags. The relevance and connectivity to vector collections and real-time communication features were also adjusted.

Issues Identified

Issue 1: Code Clarity

• Description: The SetLinks method can cause confusion as it uses inline initialization for a list.
• Suggestion: Use explicit new List<> for clarity and consistency.
• Example:

  // Before
  Links = links ?? [];
  // After
  Links = links ?? new List<AgentLink>();

Issue 2: Namespace Consistency

• Description: The namespaces changed without clear documentation of its impact; ensure consistency across the whole project.
• Suggestion: Review namespace changes in BotSharp.Core.Realtime.Websocket.Chat to ensure alignment and avoid disruption of dependencies.
• Example:

  // Confirm consistent namespace usage in all related files.

Issue 3: Magic Strings

• Description: Use of magic strings for folder paths and statuses can lead to errors and maintenance complexity.
• Suggestion: Define constants or configurations for such strings.
• Example:

  private const string AGENT_LINKS_FOLDER = "links";

Issue 4: Lack of Null Checks

• Description: Methods that read files or manipulate collection items do not consistently check for null which can lead to runtime exceptions.
• Suggestion: Implement more robust null checks.
• Example:

  if (links != null) {...}

Overall Evaluation

The code shows significant improvements in terms of new feature implementation and extending existing functionality to accommodate links similar to templates. However, making the code more maintainable and clear with better use of language features like null checks, constants for repeated strings, and following a consistent namespace structure would further enhance this submission. It's recommended to carefully review the changes with particular focus on test coverage for the new AgentLink feature and its integration with existing systems.

[GGHansome]

Auto Review Result:

Code Review Summary

Changes Summary: The changes introduced new functionalities related to AgentLink and Link fields in the codebase, including new methods for setting, getting, and registering these links. It also involved refactoring some classes to inherit from a newly created class AgentPromptBase, which centralizes common properties and methods. Additional enhancements include interface methods for handling VectorCollectionDetails and improvements in the WebSocket handling.

Identified Issues

Issue 1: [Code Consistency]

• Description: The code sometimes uses inconsistent naming conventions for similar operations, such as RegisterTag and RegisterTags, which might confuse the development team.
• Suggestion: Adjust the names to reflect the operations more consistently and document them accordingly.
• Example:

  // Consider aligning names or clearly documenting differences

Issue 2: [Null Handling]

• Description: In the SetLinks method, the null handling default to an empty array [] which is not a valid default initializer in C#.
• Suggestion: Use an empty list initializer new List<AgentLink>() for better code stability.
• Example:

  // Before
  Links = links ?? [];
  // After
  Links = links ?? new List<AgentLink>();

Issue 3: [Method Redundancy]

• Description: The Render method in TemplateRender seems to return the input template directly in cases of failure, which might not be informative.
• Suggestion: Consider differentiating this output to make debugging easier.
• Example:

  if (!FluidTemplate.TryParse(template, out IFluidTemplate t, out var error)) {
      _logger.LogWarning("Template parsing failed: {error}", error);
      return ""; // or some specific error template
  }

Overall Evaluation

The code base has been significantly enhanced to improve modularity and extensibility by introducing base classes and new interfaces. While these changes are beneficial, attention to consistent naming, correct initializer usage, and output distinction in error scenarios would further improve code clarity and maintainability. Additionally, ensure that new methods and properties are properly documented for the team to understand their intent and usage.

[GGHansome]

Auto Review Result:

Code Review Summary

Change Overview: The changes introduce new functionality and improve existing components by adding AgentLink capability, updating interfaces and classes to manage the new entity, and enhancing the template rendering system. This could aid in more flexible and extended agent behavior, increasing the system's utility.

Discovered Issues

Issue 1: Potential Code Redundancy

• Description: In several parts of the changes, the new Links functionality has been added by copying existing patterns for Templates and Functions. While this achieves consistency, it may lead to code duplication and possible redundancy.
• Suggestion: Consider utilizing a generic method to handle common functionality across similar components, reducing code duplication.
• Example:

  // Before
  public List<AgentTemplate> Templates { get; set; } = new();
  public List<AgentLink> Links { get; set; } = new();
  
  // Suggested Improvement
  public List<T> GetAgentComponents<T>() where T : AgentComponent, new() {
      return new List<T>();
  }
  

Issue 2: Missing Error Handling

• Description: The new methods such as RenderAgentLinks and RegisterTags do not have any error handling. Without this, any failure in processing could cause recursive failure through the application.
• Suggestion: Implement try-catch blocks and proper error logging within these methods.

Issue 3: Inconsistent Method Naming

• Description: The method Register was renamed to RegisterType but inconsistently in certain areas of the code.
• Suggestion: Ensure consistent method renaming and usage throughout the entire codebase to prevent misunderstandings.

Overall Evaluation

The code is generally well-structured and adheres to good practices. The addition of AgentLink functionality is a positive enhancement, promoting better data management through inheritance. Future improvements can focus on reducing redundancy by abstracting common functionalities and introducing robust error handling to increase reliability. Always aim for consistent method naming and usage across the project to enhance maintainability.

[GGHansome]

Auto Review Result:

Code Review Summary

Change Overview: The code aims to expand existing functionality by introducing a new AgentLink entity into the BotSharp project, potentially to track additional link-related information within agents. Changes across several files include the addition of new methods, properties, and classes to support this feature, as well as updates to interfaces and handlers for better session management.

Issues Found

Issue 1: Nullability Handling

• Description: The usage of Links = links ?? []; in SetLinks method can be problematic if links is meant to be a list since [] is not valid in C#.
• Suggestion: Use a default constructor or Array.Empty<>().
• Example:

  // Before
  Links = links ?? [];
  // After
  Links = links ?? new List<AgentLink>();

Issue 2: Class Initialization

• Description: The constructor of AgentPromptBase is empty, but it might be beneficial to include default values or logic if necessary.
• Suggestion: Verify if initialization logic or documentation is needed.

Overall Evaluation

The code is generally consistent and expands functionality logically. Including new methods and adjusting interfaces seem well thought, maintaining cohesion within the system. Recommended improvements involve minor adjustments for null safety and ensuring class initializations are clear and concise. Always ensure proper handling of null values and maintain code clarity with appropriate initializations.

[GGHansome]

Auto Review Result:

Code Review Summary

Change Overview: The code changes introduced a new concept/model "AgentLink" to the BotSharp project, enhancing the functionality by allowing linkage in agent prompts. This modification is intended to enrich how agents can be configured and interact with templates and links, improving flexibility and customization.

Identified Issues

Issue 1: [Code Clarity]

• Description: There are instances of non-descriptive variable names and commented out code. For example, Combine(WaveFormat) is commented but left in the final code, which could cause confusion.
• Suggestion: Remove commented out code unless it is intended to be used in future revisions with appropriate comment detailing its future use.
• Example:

  - //_bufferedWaveProvider.BufferLength = 1024;
  + // If necessary for future buffer length adjustment, uncomment the line below and provide rationale.
  

Issue 2: [Null Safety and Default Initialization]

• Description: The new SetLinks method assigns an empty list when null but uses [], which is not valid C# for list initialization.
• Suggestion: Use new List<>() for consistency and clarity.
• Example:

  public Agent SetLinks(List<AgentLink> links)
  {
      Links = links ?? new List<AgentLink>();
  

Issue 3: [Interface and Method Signature Consistency]

• Description: Changes in interfaces like adding parameters should be reflected in all implementing classes and usages. E.g., OnSessionUpdated addition of isInit might cause issues if not handled in all implementations.
• Suggestion: Review implementing classes and ensure all such changes are propagated and accounted for.

Issue 4: [Adherence to C# Coding Standards]

• Description: Some methods and variables deviate from .NET naming conventions, such as prefixed underscores for private fields or camelCase for method parameters.
• Suggestion: Follow general C# naming guidelines for clarity.

Overall Evaluation

The code introduces valuable enhancements to the agent infrastructure, particularly with the flexible handling of links within templates. However, attention needs to be paid to coding standards, clarity, and consistency, especially across interfaces and method signatures, to maintain code quality and prevent integration issues.

[GGHansome]

Auto Review Result:

Code Review Summary

Summary of Changes: The submitted code introduces several new features and updates to existing code. The main changes include the addition of a Link field in the AgentField enum and associated functionality in the Agent class, the creation of new classes AgentLink and AgentPromptBase, updates to a number of interfaces and service classes, and enhancements to the real-time and websocket components. These changes aim to enhance the agent capabilities by integrating a new data type (Link) which allows agents to handle links more seamlessly and provide better real-time communication and data handling capabilities.

Issues Identified

Issue 1: [Code Quality]

• Description: The code contains some redundant initializations and inconsistent code styling, such as unnecessary default constructors and strings not being validated or sanitized for potential security issues.
• Suggestion: Remove redundant constructors unless needed for specific reasons, and ensure any string inputs are validated or sanitized to prevent possible security vulnerabilities.
• Example:

  // Before
  public class AgentLink : AgentPromptBase
  {
      public AgentLink() : base() {}
  }
  
  // After (Remove if there's no specific need)
  public class AgentLink : AgentPromptBase
  {
  }

Issue 2: [Potential Null Reference]

• Description: Usage of the null conditional (??) operator without safeguarding might lead to unintended results if default behavior isn't desired.
• Suggestion: Consider explicit null checks or ensuring meaningful defaults to prevent unexpected behavior.
• Example:

  public Agent SetLinks(List<AgentLink> links)
  {
      Links = links ?? [];
      return this;
  }
  // Should verify the desired behavior of empty list assignment

Overall Evaluation

The code changes are generally well-structured and aim to enhance functionality, especially in handling and extending agent and template capabilities. However, attention should be paid to code styling consistency, redundancy, and potential security pitfalls involving string handling. Additionally, clear handling of null values and consistent commenting for public methods will improve maintainability and clarity going forward.