Tracking AI footage is cursed because there is no real camera and no real parallax

by @harperverma_vfx970 in c/vfx-breakdowns · · Tracking / Matchmove

Tried to matchmove a CG element onto an AI 'drone' shot. The tracker exploded. Why? Because the AI camera doesn't obey real perspective - parallax is inconsistent frame to frame, so the solver can't find a coherent 3D camera. The 'camera' is a vibe, not a physical object.

My workaround: don't 3D track. Do a 2.5D projection or straight 2D planar track on a local region and parent your element to that. Stop asking a geometry solver to reverse-engineer a camera that was never real.

153 points · 8 comments

Comments

  • @oscarmiller_art · · 75 points

    2.5D projection is the way. Treat it like matte painting projection, not like a tracked plate. You're faking depth onto a thing with no real depth.

  • @emmapatel_creates · · 98 points

    'the camera is a vibe, not a physical object' should be the sub banner. The number of hours I lost before accepting AI footage will NOT 3D solve.

  • @masonjohnson_ai6884 · · 88 points

    I spent a full day trying to get a clean solve on a Kling push-in before I realized the parallax literally reverses in the back third of the shot. no solver survives that.

  • @victorsharmamedia610 · · 24 points

    Controversial: I think trying to put hard-surface CG into AI plates is a losing battle and we should comp AI-into-AI or CG-into-real and stop mixing the two paradigms.

  • @alexandergupta_art677 · · 94 points

    Disagree hard. 'CG hero prop into AI environment' is the most commercially useful combo there is. The env is cheap from AI, the hero needs CG precision. That's the future, not a mistake.