Made a full performance MV with a consistent AI singer, she holds across 40 shots, lip sync is real

by @ravilee_motion569 in c/music-videos · · Performance

Made a full performance MV with a consistent AI singer, she holds across 40 shots, lip sync is real
The hard part of an AI performance video is character consistency AND lip sync across a lot of cuts. Soul ID for the face, a dedicated lip-sync pass per shot. Breakdown and the sync tool I used in comments.
youtube.com

The hard part of an AI performance video is character consistency AND lip sync across a lot of cuts. Soul ID for the face, a dedicated lip-sync pass per shot. Breakdown and the sync tool I used in comments.

663 points · 10 comments

Comments

  • @takeshiharris_digital · · 98 points

    the consistency across 40 shots is genuinely the best i've seen in an AI performance piece. she's the SAME person in the close-up and the wide, which almost never holds. the face didn't drift once. how.

  • @nikolaitanaka_art · · 84 points

    Soul ID locked from a single master portrait, and crucially i never let the model generate her face 'fresh' per shot, every shot inherits the identity embedding plus a reference crop. the second you let it improvise the face, she becomes her own cousin.

  • @felixpatellab7922 · · 35 points

    she has actual STAGE PRESENCE which is wild for a person who doesn't exist, the way she leans into the camera on the chorus. did you direct that or did the model do it

  • @hiroshirodriguezpro822 · · 31 points

    directed via the motion prompt, 'leans toward camera, intense eye contact, chin slightly down' on the chorus shots. you can direct an AI performer, most people just don't write performance direction into the prompt.

  • @lucasguptapro335 · · 42 points

    lip sync is clean but a couple shots have that slight 'puppet jaw' where the lower face moves but the cheeks don't follow. which sync tool, and did you do any manual cleanup?

  • @raviwilliamscreative · · 38 points

    yeah the puppet jaw is the current ceiling of automated sync, it moves the mouth but doesn't drive the whole-face coarticulation. i picked the shots where she's further from camera for the busiest lyrics to hide it, and saved the clean close-ups for sustained notes.