FaceForge
Turn spoken audio into facial animation — solve, retarget, bake.
Turns spoken audio into facial animation: solves a sound wave to face-control curves, retargets those curves into whatever vocabulary a character's face understands, and bakes the result to an animation sequence and a montage.
It knows nothing about any particular solver, and nothing about any gameplay framework. It ships with no solvers of its own.
FaceForge is the last third of the dialogue pipeline. It never asks where its audio came from, which is precisely why a synthesised line and a recorded performance reach it the same way.
| Plugin | Role | Licence | Version |
|---|---|---|---|
| FaceForge source | coreThe subsystem. Install this one. | Open | 0.2.2 |
| FaceForgeACE | providerA route to a generator — a vendor API, or a container on your own GPU. | Free | 0.2.2 |
| FaceForgeACEToolset source | toolsetThe same subsystem as typed MCP tools, for an agent. Adds nothing of its own. | Open | 0.1.1 |
| FaceForgeMetaHuman source | adapterBridges the core into a framework you already build on. | Open | 0.2.1 |
| FaceForgeToolset source | toolsetThe same subsystem as typed MCP tools, for an agent. Adds nothing of its own. | Open | 0.2.2 |
Solve and bake are separate, deliberately
| Stage | Cost | Bound to |
|---|---|---|
| Solve — audio to curves | Slow | The provider |
| Bake — curves to an animation a character can play | Milliseconds | The character |
Clips keep their solved curves permanently. So pointing a second character at the same audio is a re-bake, not a re-solve.
This is the single most useful property of the set. Face solving is character-agnostic — the solve is a function from audio to named floats, and only the vocabulary those names are in is character-specific.
A line solved once can dress every character who says it.
Layers
Several whole solves kept side by side on one clip, merged by a recipe stored as data on the clip: the first layer as a base, later layers laid over it at their own weight.
Merging is milliseconds and repeatable.
That is what makes the interesting combinations possible — a raw video solve as the base with a mouth-only overlay on top, A/B-proven to change the bake as the overlay's weight changes.
That combination is the one recorded performances take. In the demo scene, the two lines that were performed on camera read "Video + Mouth" in the Face Bank panel's Layers column and name both sources — the video take and its audio — in "Solved from". The other five, generated from text, have no layers at all.
See the panel.
Hand correction, without losing the solve
An animator's fix and a re-solve are usually mutually exclusive. Here they are not:
Build a correction sequence
A Level Sequence with the solve keyed on an absolute base section and an empty additive section on top.
Key only the additive section
The animator works normally. The base is read-only underneath.
Snapshot it back
The additive section is read back as offsets into a Corrective layer.
The base solve can now be re-run underneath the hand fix without discarding it. Verified frame-exact.
See correction.
Graduation
Two axes — status and origin — the same shape as
SpeechForge.
Anything whose origin is not Generated is left alone by the pipeline.
Where to go next
Concepts
Face banks, clips, and the preview stage.
Vocabularies
Why a mismatch is made inexpressible, and the two kinds of transform.
Providers
The engine's own solver, and NVIDIA Audio2Face-3D.
Correction
The round trip an animator can use without blocking a re-solve.
The toolset
What an agent can do here.