Handling Citations and Tool Explosion Issues

[Willem (wct432)]

Hi, I had two questions regarding Semantic Kernel which I hope could be useful for others as well.

1. How can I build a list of citations that inform the user of where the copilot received its answer like we see in Bing Copilot? I wasn't sure if this functionality is incorporated in SK, or if it something developers need to implement with custom logic. I couldn't find an easy way to aggregate the data from the plugins.
2. How does Semantic Kernel handle tool explosion? In testing I've done with other frameworks you can realistically only represent a limited number of tools to the LLM before you start butting up against issues to do with context window. How does SK manage this?

Thanks in advance for your help!

[Frederik Munster (solidaris-frederikmunster)]

I'm also trying to retrieve the citations, but seems not possible with the StreamingChatMessageContent.

await foreach (StreamingChatMessageContent updates in chatCompletionService.GetStreamingChatMessageContentsAsync(history, openAIPromptExecutionSettings, kernel)) { if (!string.IsNullOrEmpty(updates.Content)) { currentAnswer += updates.Content; StateHasChanged(); } }

Where as it is possible with the StreamingChatCompletionsUpdate object

await foreach (StreamingChatCompletionsUpdate chatUpdate in openAIClient.GetChatCompletionsStreaming(completionOptions)) { //Show answer if (!string.IsNullOrEmpty(chatUpdate.ContentUpdate)) { assistant_response += chatUpdate.ContentUpdate; Console.Write(chatUpdate.ContentUpdate); } if(chatUpdate.AzureExtensionsContext != null && chatUpdate.AzureExtensionsContext.Citations != null) { foreach (AzureChatExtensionDataSourceResponseCitation citation in chatUpdate.AzureExtensionsContext.Citations) { Citations.Add(citation); } } }

[Sophia Lagerkrans-Pandey (sophialagerkranspandey)]

Willem (@wct432) We recommend you asking the AI to provide citations in the prompt. The second person is using Azure OpenAI with your data which comes with citation with data. This should help you as well. For your second question, we've also included this sample for your reference as well: https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Optimization/PluginSelectionWithFilters.cs

Frederik Munster (@solidaris-frederikmunster) for your issue, can you create an issue on this so we can triage and get this fixed?

[reckitt-gregpivniev]

This is not really a solution.
In complex scenarios, LLM often tend to hallucinate references, and citations are often corrupted.

[SiddarthaCherukumudi]

Hello Sophia Lagerkrans-Pandey (@sophialagerkranspandey) ,
Sorry for repeating the question. Can you please clarify that GetStreamingChatMessageContentsAsync cannot provide Citations like GetChatMessageContentAsync? I have also noticed that asking to provide references in the prompt regarding questions is giving garbage values. Can you please provide the prompt which would help us to get the proper citation?