Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Add a graph named "voice_assistant_multimodal" in the property.son of default. #589

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

zhuermu
Copy link

@zhuermu zhuermu commented Feb 13, 2025

Some developers are confused about debugging the Amazon Nova multimodal model, so we are adding a copy of the demo graph to the default configuration.

@cyfyifanchen
Copy link
Collaborator

LGTM

@cyfyifanchen cyfyifanchen self-requested a review February 13, 2025 05:55
Copy link
Collaborator

@cyfyifanchen cyfyifanchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@plutoless plutoless left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should register nova as a llm module in playground moduleConfig.ts
the concept of playground is using the same connections while developers are able to switch modules. if you believe video feature is needed, it should rather be a option feature flag for llm module.
to conclude,

  1. add nova to llm registry in moduleConfig.ts
  2. in moduleConfig.ts for llm module add an option modalities=["image", "text"]
  3. change ChatCfgModuleSelect.tsx so that a connection from rtc to llm will be added when a llm module accepts image input

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants