NVIDIA ACE is a suite of technologies that help developers bring digital humans to life with generative AI. ACE NIMs are microservices designed to run in the cloud or on PC.
On this Git repo, you will find samples and reference applications using ACE NIMs and microservices. However, these microservices can be obtained through an evaluation license of NV AI Enterprise(NVAIE) through NGC.
| Technology | Description | Software Support | Cloud Deployment | Windows Deployment |
|---|---|---|---|---|
| Riva Automatic Speech Recognition | Speech -> Text | NVIDIA AI Enterprise | X | Coming Soon |
| Riva Neural Machine Translation | Text Translation | NVIDIA AI Enterprise | X | |
| Riva Text-to-Speech | Text -> Speech | NVIDIA AI Enterprise | X | Coming Soon |
| Audio2Face | Audio -> Blendshapes for Facial Lip-sync | NVIDIA AI Enterprise | X | Coming Soon |
| AnimGraph | Animation controller | NVIDIA AI Enterprise | X | |
| Omniverse RTX Rendering Microservice | Omniverse Based Pixel Streamer | NVIDIA AI Enterprise | X | |
| ACE Agent | Conversational Controller, RAG Workflows | NVIDIA AI Enterprise | X | |
| Maxine Speech Live Portrait | 2D Picture Lipsync and Animation | Early Access Evaluation | X | |
| Nemotron-3 4.5B SLM | Small Language Model | Early Access Evaluation | Coming Soon | X |
| Gaming Reference Workflow | Audio2Face Unreal Engine Examples | Example Workflow | X | Coming Soon |
| Customer Service Reference Workflow | Full reference workflow of customer service and kiosk usecases | Example Workflow | X |
NVIDIA pre-trained models provide industry-leading quality and real-time performance.
AI models trained on commercially safe, responsibly licensed data. Fine-tuning and guardrails enable accurate, appropriate, and on-topic results no matter the user's input.
Handle inference through any public or private cloud, Windows PC, or a mix of both.
Developers can leverage ACE to build their own digital human solutions from the ground up, or use NVIDIA's suite of domain-specific AI workflows for next-generation non-playable game characters (NPCs), interactive digital assistants for customer service, and digital avatars for real-time communication.
NVIDIA Kairos Sample showcases an easy to use Unreal Engine project using the Audio2Face microservice. This sample shows how to connect Audio2Face to Metahuman and configure the Audio2Face microserivce.
Learn More About ACE NIMs for Gaming
NVIDIA Tokkio is a digital assistant workflow built with ACE, bringing AI-powered customer service capabilities to healthcare, financial services, and retail. It comes to life using state-of-the-art real-time language, speech, and animation generative AI models alongside retrieval augmented generation (RAG) to convey specific and up-to-date information to customers.
Learn More Tokkio Customer Service Workflow
Full ACE developer documenation
| Component | Documentation | Video/Tutorial |
|---|---|---|
| Getting Started | NVIDIA Docker Setup, Install Kubernetes | |
| NVIDIA UCS | Documentation | |
| NVIDIA Audio2Face | Documentation | Coming soon! |
| NVIDIA Riva ASR | Documentation | Coming soon! |
| NVIDIA Riva TTS | Documentation | Coming soon! |
| NVIDIA Riva NMT | Documentation | Coming soon! |
| NVIDIA ACE Agent Microservices | Documentation | Coming soon! |
| NVIDIA Maxine Live Portrait | Documentation | Coming soon! |
| NVIDIA Avatar Configurator & Avatar Customization | Documentation | Coming soon! |
| NVIDIA Animation Graph Microservice | Documentation | Coming soon! |
| NVIDIA Omniverse Renderer Microservice | Documentation | Coming soon! |
| Example | Description | Video |
|---|---|---|
| Text-to-Gesture | Text-to-Gesture using A2X & Animation Graph Microservices | Creation of Basic Sentiment Analysis Utility, Connecting all Microservices in UCF, Deployment & App Execution |
| Reallusion Character | Exporting Character in Reallusion Character Creator + Audio2Face | Exporting Character from Reallusion Character Creator & Preparing Character in Audio2Face, Setup, streaming through a Reference App & Fine Tuning |
| Stylised Avatar | Building Stylised Avatar Pipeline with ACE Components | Making & Animating a Stylised 3D Avatar From Text Inputs, Make Vincent Rig Compatible for UE5 & A2X LiveLink, Make Vincent Blueprint Receive A2X Animation Data, Create Python App to Generate Audio from Text & Animate Vincent |
Github - Apache 2
ACE NIMs and NGC Microservices - NVIDIA AI Product License
