Posted 2/29/2024, 10:29:27 PM
Alibaba's New AI Generates Realistic Facial Animations from Audio
- Alibaba unveiled an AI video generator called EMO that creates realistic facial animations from audio clips
- EMO was showcased making the Sora lady from AI Tokyo sing and emote a Dua Lipa song convincingly
- It can also make a still image of Audrey Hepburn credibly speak and emote audio of Lili Reinhart talking
- The technology seems more advanced than previous audio-to-facial animation systems like NVIDIA's Audio2Face
- EMO matches facial animations to audio through a reference-attention mechanism while retaining the face's original features