Google Unveils Lumiere AI to Generate Strikingly Realistic Video
-
Google's new Lumiere AI video model creates more realistic and seamless video compared to other AI video generators like Runway. It uses a new framework called Space-Time-U-Net (STUNet) to focus on object movement.
-
Lumiere generates 80 frames compared to 25 frames from Stable Diffusion Video, allowing for smoother motion. Example videos show significant improvement over previous AI video quality.
-
Professional video editor noted Lumiere intros could pass for CGI if not told it was AI, though some artificiality remains visible on close inspection.
-
STUNet allows Lumiere to generate content based on where things should be at a given time rather than stitching together individual frames. This enables more fluid motion.
-
Beyond text/image-to-video, Lumiere capabilities will include stylized video, cinemagraphs, inpainting. But authors note concerns around potential misuse.