YouTube is piloting AI lip-sync that aligns mouth movements with translated audio, expanding auto-dubbing. Limited to 1080p, 5 languages, and select creators.
© E. Vartanyan
YouTube is pressing ahead with AI-driven tools to lift viewer engagement. After rolling out automatic dubbing, the platform is now piloting a next step: syncing lip movements to the translated audio track. It’s a logical evolution that aims to make dubbed videos feel less like add-ons and more like native productions.
According to Auto-dubbing lead Buddika Kottahachchi, the technology relies on specialized algorithms that adjust mouth shapes and facial micro-movements at the pixel level so the result looks as natural as possible. The system factors in the three-dimensional geometry of lips, teeth, and facial expressions, and is powered by a custom AI model developed by Google.
For now, the feature works only with videos up to 1080p and supports five languages: English, French, German, Portuguese, and Spanish. Google plans to grow that lineup to more than 20 languages, including all those already available in YouTube’s auto-dubbing. The narrow scope signals a cautious rollout while the tech matures.
The launch is limited to a small group of creators, much like the early days of auto-dubbing. Wider access is expected later, and creators will be able to decide whether to enable lip sync for their videos. There is talk the option could be paid, though it remains unclear who would cover the cost—channel owners or viewers. If the execution matches the promise, audiences may spend less time noticing the dub and more time with the story.