Repository navigation
Bedrock Claude Error: roles must alternate – Works Locally with Ollama #6608
Replies: 1 comment
|
Thanks for pasting the full Short version: Bedrock's Claude 3 Haiku endpoint rejects two consecutive
from typing import Any
from autogen_ext.models.anthropic import AnthropicBedrockChatCompletionClient
def _blocks(content: Any) -> list[Any]:
return [{"type": "text", "text": content}] if isinstance(content, str) else list(content)
def merge_same_role_turns(messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
merged: list[dict[str, Any]] = []
for m in messages:
if merged and merged[-1]["role"] == m["role"]:
merged[-1] = {"role": m["role"], "content": _blocks(merged[-1]["content"]) + _blocks(m["content"])}
else:
merged.append(dict(m))
return merged
class MergedTurnsBedrockClient(AnthropicBedrockChatCompletionClient):
"""Same client, but adjacent same-role turns are merged before the request goes to Bedrock."""
def __init__(self, **kwargs: Any) -> None:
super().__init__(**kwargs)
create = self._client.messages.create
async def patched(**kw: Any) -> Any:
kw["messages"] = merge_same_role_turns(kw["messages"])
return await create(**kw)
self._client.messages.create = patched # type: ignore[method-assign]Merging at the request level matters once an agent calls a tool: its context becomes Code path and raw SDK checks
Raw
|
Uh oh!
There was an error while loading. Please reload this page.
I am trying to get this workflow to run but getting this error.
I can read and see what the issue is but have no idea as to how I can prevent this. This works fine with some other issues if ran with a local ollama model. But with Bedrock Claude I am not able to get this to work.
Any ideas as to how I can fix this?
All reactions