cache_seed in GroupChat

[mkojima123]

When setting a cache_seed, the following code always returns the same response:

result = agent1.initiate_chat(agent2, message="tell me a joke.", max_turns=2)

However, when using a GroupChat (where a cache_seed is set for all agents), the response changes on each execution:

result = agent1.initiate_chat(groupchatmanager, message="tell me a joke.", max_turns=2)

Questions

1. Is this the expected behavior?
2. How can I make GroupChat return the same response consistently when a seed is set?

Thank you.

[Eric Zhu (ekzhu)]

The group chat manager's LLM config must be set with a seed also.

Another thing: the seed is for LLM caching but doesn't prevent the code execution or function calls returning different results. That can cause cache misses as well.

[mkojima123]

Thank you for your response.

I have set cache_seed in llm_config for GroupChat as well, but the responses still vary with each execution.
Is it inherently difficult for GroupChat to produce the same interactions?
Thank you.

[Eric Zhu (ekzhu)]

Yes. as long as you have other sources of input like function calls and code execution, it is hard. Unless you cache in those input sources as well.

[mkojima123]

So, unless all inputs are exactly the same, it is difficult, right?
I understand clearly. Thank you!

[Eric Zhu (ekzhu)]

Yep!

It looks like you are using v0.2 version of the API. Would be great if we can have a solution for this in the latest v0.4 version. For example. Caching for tool calls and code executors.

[mkojima123]

I can't update to 0.4 right away, as it may affect other libraries, but I would love to try it when I have the opportunity.
Thank you for the helpful information!