Can MCP Chunk the files?

[mohamedmansour]

I am using the Azure.AI.OpenAI client, and trying out this MCP C# SDK, and I am getting this error:

Invalid 'messages[4].content': string too long. Expected a string with maximum length 1048576, but got a string with length 2045715 instead.
   at Azure.AI.OpenAI.ClientPipelineExtensions.ProcessMessageAsync(ClientPipeline pipeline, PipelineMessage message, RequestOptions options)

How would I go ahead and fix the chunking boundary/chunk construction?

[stephentoub]

Can you share a standalone repro? How are you constructing the messages?

[mohamedmansour]

That was quick :D I am just trying out this SDK today!

I am so far loving this SDK it is so easy to use and write tools! Thank you! So far I wrote these tools:

Tools available:
  FindReferences
  FindDefinition
  GetClangAST
  RunClangTidy
  FormatCode
  ExtractClassStructure
  FindIncludes
  GitGrepSearch
  GitDiff
  GitLog
  ReadFile
  WriteFile
  SearchInFile
  ListFiles
  GetFileInfo
  CopyFile
  CreateDirectory
  ExtractCodeStructure

But used the stock sample app to do the chat/MCP integration, I know it is my fault cause that is very basic thing to do with o3-mini, but would be very nice if the SDK had this utility of chunking for me, for example I can give it the model and context length perhaps as config:

AzureOpenAIClient azureClient = new(
    new Uri("https://......openai.azure.com"),
    credential);
using IChatClient chatClient = azureClient.AsChatClient("o3-mini")
    .AsBuilder().UseFunctionInvocation().Build();

List<ChatMessage> messages = [];
while (true)
{
    Console.Write("Q: ");
    messages.Add(new(ChatRole.User, Console.ReadLine()));

    List<ChatResponseUpdate> updates = [];
    await foreach (var update in chatClient.GetStreamingResponseAsync(messages, new() { Tools = [.. tools] }))
    {
        Console.Write(update);
        updates.Add(update);
    }
    Console.WriteLine();

    messages.AddMessages(updates);
}

[stephentoub]

Thanks.

I think there are likely two aspects here.

There's the length limitation OpenAI imposes on the length of an individual message. That could be addressed by splitting the single message up into multiple messages, as you say.

But there's the more challenging aspect of token limitation. If you're sending a 1mb message, that's likely several hundred thousand tokens, which is very likely going to exceed the context window of your model. No amount of splitting up the tool result message into multiple messages will help with that.

Both of these can be addressed external to the MCP library by plugging in a custom IChatClient after the FunctionInvokingChatClient that culls back the results from the tool. But it'd be really hard for a general-purpose middleware filter to do so on an arbitrary tool result; it'd be removing content from that tool result, and it's not clear how it could do so in a way that would do the "right thing".

It'd be better if the tool didn't return 1mb of content.

cc: @SteveSandersonMS