A talking avatar starts with a simple idea: take a picture of a character, give it something to say, and bring it to life. The interesting part is making those pieces work together. The character needs to stay recognizable, its mouth needs to follow the speech, and its movement needs to feel like part of the same performance.
This project explores that through a trained adapter for MiniMax-H3 Ref2VA, a local interface, and a reference serving setup. It takes a single image and a short line of dialogue and generates a talking-avatar clip.
What I built
The interface asks for four things: a reference image, dialogue, scene direction, and a background. Direction can describe expression, movement, or camera behavior. The background can come from a preset or a custom description.
I kept the model recipe behind the interface. Someone trying a character should be able to focus on what it says and how it behaves without choosing checkpoints, inference steps, or audio modes. The repository packages the selected adapter together with the training code, evaluation reports, and inference configuration.
Adapting the model
The base is MiniMax-H3 Ref2VA. Rather than training the entire model from scratch, this project uses a LoRA adapter with rank 16 and alpha 32. It targets attention and feed-forward modules and has 10,655,744 trainable parameters.
The documented training run uses 481 accepted transcript-conditioned clips, of which 385 are training clips. Training runs for 1,000 optimizer steps, or about 2.6 passes through the training set. The cache uses 384 × 384 images and 56-frame sequences.
These details matter because this is a specific adaptation of an existing model. The trainable parameter count describes the adapter, not the full base model. Similarly, a short training run on this dataset is not evidence that the model will generalize to every character or speaking style.
From checkpoint to something usable
A trained checkpoint is only one part of a usable demo. The project also includes a local UI, a Modal reference endpoint, and a partner bundle with adapter weights, configuration, and an example prompt.
The inputs are deliberately consistent across the UI and hosting interface. A reference image establishes the character, dialogue provides the spoken content, and direction describes the performance. That makes the model easier to try and easier to integrate into another experience.
The repository keeps evaluation prompts, gates, benchmark reports, and checkpoint inspection notes alongside the implementation. Those are the places to inspect the behavior of the selected model in more detail.
The serving constraint
The adapter is small compared with the base model, but that does not make the complete system cheap to start. The serving challenge is keeping the H3 model loaded and ready for inference.
The Modal endpoint is a reference implementation. The repository recommends a dedicated inference provider that keeps the base model warm for production hosting. This is a useful distinction between demonstrating a model and operating it as an interactive product.
Why I’m interested in this
A speaking character gives an interactive world a visible presence. It is also a focused way to explore continuity: the face, movement, and speech all need to belong to the same character over time.
This project covers the generation and serving part of that problem. Persistent memory, goals, and an ongoing conversation would require additional systems around it. A convincing generated clip is a starting point for that larger experience.