Runway Adds Realistic Lip Sync to AI-Generated Characters
-
Runway has added a new lip sync feature that animates mouth and face movements to match audio clips. This brings more realism and interactivity to AI-generated video characters.
-
The lip sync feature was tested on a variety of AI-generated character images, using both synthetic and real voice recordings. The quality of the lip sync animation improved realism.
-
The lip sync handled text-to-speech better than real audio in some cases. Quality depended partially on the rendering style of the character image. More photorealistic images showed some uncanny valley effects.
-
Runway’s lip sync compares well to similar commercial tools, with good mouth movement that is in sync with longer audio clips. The added head motion increases realism.
-
While not perfect, the new lip sync capability expands the use cases for AI video generation significantly. It allows for conversing characters rather than just static images.