AdvisedRequest removed with RC1

[uwilken]

Hi,

I am working on a project where we share data between Tools and RequestAdvisors.

Tools add data to the tool context and an Advisor uses this data to add additional annotations to the final answer.

With #2993 the class AdvisedRequest has been removed. Instead, ChatClientRequests are used now in Advisors.

In ChatClientRequest it is possible to access the context that is available to all other Advisors, but there is no way to access the tool context (formerly AdvisedRequest#toolContext).

@markpollack, @ThomasVitale, Is there a way to access the tool context in an Advisor?

Your feedback is much appreciated!
Ulrich

[ThomasVitale]

@uwilken the ChatClientRequest gives you access to the fully formed Prompt now, from which you can retrieve the tool context.

ChatOptions chatOptions = chatClientRequest.prompt().getOptions();
if (chatOptions instanceof ToolCallingChatOptions toolCallingChatOptions) {
	Map<String, Object> toolContext = toolCallingChatOptions.getToolContext();
}

I hope that helps!

[ThomasVitale]

@uwilken If I understood correctly, you add new items to the ToolContext from within a tool, and then you want to read them back from an Advisor. Did I understand that correctly?

Did you have this use case working correctly with the previous AdvisedRequest API? I'm asking because, if I'm not wrong, the ToolContext has always been a uni-directional channel to pass context from a chat model/client to the tool execution (without the possibility to go the other way around).

[uwilken]

@ThomasVitale Yes, with AdvisedRequest#toolConext we can access data that has been added to the ToolContext during tool execution. This works with version 1.0.0-M8.

If the ToolContext is not meant to work this way around, is there another way to collect data during tool execution and pass it on so it can be consumed later on in com.coremedia.ai.kio.backend.advisors.RequestAdvisor#after?

[tzolov]

@uwilken, AFAIK, ToolContext wasn't designed to return data from tools to the application, though we didn't block this functionality.
I understand how this can be an important use case (especially since SpringAI controls tool execution by default).
I can't think of workaround currently, but we'll address this regression of an unintentional feature in version 1.0.1.

[uwilken]

Hi @tzolov,
thank you for your feedback. This function is crucial for our application and I could imagine it being useful for others as well. I will stick with 1.0.0-M8 for now and hope that 1.0.1 will support it again.
Do you have a rough time estimate for 1.0.1?

[uwilken]

We found a simple workaround.
We just had to add a modifiable map to the initial tool context of the chat client. This one can be accessed and modified in Tools and RequestAdvisors later.

No need to change the API.
Thanks for your help!