text input from datastream for v1.0 #1521
Conversation
…el-user-transcription
|
| extra={"text": text, "participant": self._participant_identity}, | ||
| ) | ||
| self._agent.interrupt() | ||
| self._agent.generate_reply(user_input=text) |
There was a problem hiding this comment.
we are calling agent inside the RoomInput, is that okay? should the agent.input has a text input, or we make agent required for RoomInput?
There was a problem hiding this comment.
Ok, I'll do that when merging the RoomInput and Output.
| @@ -48,6 +50,7 @@ class RoomOutputOptions: | |||
| DEFAULT_ROOM_INPUT_OPTIONS = RoomInputOptions() | |||
| DEFAULT_ROOM_OUTPUT_OPTIONS = RoomOutputOptions() | |||
| LK_PUBLISH_FOR_ATTR = "lk.publish_for" | |||
| LK_TEXT_INPUT_TOPIC = "lk.room_text_input" | |||
There was a problem hiding this comment.
we should be clear about what topics we want to support. If the goal is to have this work ootb with chat components, then choosing a custom topic here might not be the best choice
There was a problem hiding this comment.
I think the question is are we going to keep the chat components in python/js sdk, and having both the original and the datastream or only the datastream for it? If so I can adjust here accordingly. cc @davidzhao
There was a problem hiding this comment.
the chat components are for client-side. but Python/Node agents should agree on the same topic so that it works with the client-side components
@lukasIO what do you recommend we should use? is client-side component listening to both the transcription topic and chat?
There was a problem hiding this comment.
currently chat components only listen to the chat topic and also send their messages only on the chat topic
There was a problem hiding this comment.
how would this work with how we are sending transcriptions? do you suggest also sending transcriptions to chat topic?
There was a problem hiding this comment.
that was my understanding, yes. But maybe I misunderstood or am forgetting something.
Why wouldn't you want it on the chat topic?
There was a problem hiding this comment.
mostly wondering if there's any conflicts between what the agent would want to use as input.. versus what is being spit out as output.
i.e. if there are two agents in the room, would that cause any cross talk.. or if the agent is being added to a livestream with a chat feature, would it automatically start interpreting random transcripts.
for that reason it seems it might be a good idea to be explicit about what is being sent to the agent as input?
|
Do we wan to keep the ChatManager in python sdk, related PR livekit/python-sdks#360? |
IMO we should deprecate and remove ChatManager.. agents should not use it any longer |
No description provided.