Behind the Scenes: I built a REAL clay set with my hands, then populated it with AI characters. The hybrid is the sweet spot.

by @morgandavis_art in c/claymation · · Behind the Scenes

After months of fighting the model to produce convincing clay environments, I gave up and just... made the set out of real clay on my kitchen table. Then composited AI-animated characters into it. This hybrid approach solved more problems than any pure-AI technique.

Why real set + AI characters works so well:
The real clay set brings GENUINE texture, real thumbprints, real imperfect lighting interaction that the model can never fully fake. The world is authentically handmade because it IS.
The AI handles the CHARACTERS — the hard part, the performance, the squash-and-stretch animation that would take me weeks to do in real stop-motion frame by frame.
The character inherits the set's lighting, so compositing is believable, and crucially the character picks up the set's REAL texture language so it reads as belonging to the same handmade world.

The trick to selling it: match the AI character's frame rate to real stop-motion (on twos), add contact shadows where they touch the real set, and run a unified grain over the whole composite so the real and the AI share one 'film stock'. The grain marries them.

It's the best of both: real-world tactility, AI-powered performance. Photos of the real set + the final composite in comments. My kitchen is covered in plasticine and I regret nothing.

1,188 points · 7 comments

Comments

  • @masondavis7942 · · 97 points

    'the grain marries them' is the compositing secret of all time. a unified grain pass over a real+AI composite is what sells the whole lie. the eye accepts that everything sharing one film-stock texture belongs together. it's how VFX has married CG to live action for decades.

  • @jordansingh_vfx6767 · · 54 points

    grain is the great equalizer. real plate + CG + AI, run one grain over all of it and suddenly it's one image. the number of composites saved by a final grain pass is uncountable.

  • @sashashah_motion389 · · 71 points

    'my kitchen is covered in plasticine and I regret nothing' is the most claymation sentence ever posted. this is the spirit. get your hands dirty, let the machine do the walk cycles.

  • @victorjohnson_vfx · · 99 points

    building the REAL set and only using AI for the characters is genuinely the smartest hybrid workflow I've seen here. you put the human labor where it has the highest payoff (real tactile texture) and the AI where IT has the highest payoff (tedious frame-by-frame performance). perfect division of labor.

  • @kaisantos1212 · · 100 points

    this inverts what most people do (AI everything, composite nothing) and it's so much better. a real handmade set has a texture DEPTH that no model matches, and sets are the EASY part to make by hand — it's the character performance that's brutal in real stop-motion. let the machine do the 700 frames of walk cycle, you make the cozy little table.