Sky replace on AI footage that already had a sky: you have to DELETE a sky before you add one

by @avadavis_ai in c/vfx-breakdowns · · Sky Replace

Weird problem unique to our corner of VFX. On real plates you shoot blue/overcast and replace. On AI footage the model already baked in a sky, often a gorgeous one, but it's the WRONG sky and it's lighting the whole scene.

You can't just matte over it - the old sky's color is bouncing off every surface. I ended up having to neutralize the scene's color cast (desaturate toward grey in the highlights), THEN relight to the new sky's key. Basically un-baking the AI's lighting decision.

This is the part nobody warns you about with 'just regenerate it.' Sometimes the comp is faster than the reroll.

116 points · 8 comments

Comments

  • @jinweber_ai680 · · 100 points

    'un-baking the AI's lighting decision' is the entire job description of comping AI footage and I've never seen it said so cleanly.

  • @victorsinghcreative · · 92 points

    Genuinely might put 'lighting-decision un-baker' on my reel. Half my shot time is talking the model out of a choice it already committed to.

  • @yukimueller_art · · 44 points

    desaturating toward grey in the highlights specifically (not the whole image) is the pro move here, good callout

  • @ethanwrightworks717 · · 65 points

    Counterpoint from the reroll camp: for a 3-second clip, 8 rerolls is 20 minutes. Your un-bake is an hour. Sometimes the slot machine wins.

  • @sakurasharma_fx · · 93 points

    True for 3 seconds. Try it on a 12-second continuous move where the reroll also changes the character's face. Now the comp is the cheap option.