Behind the Scenes: I built a REAL clay set with my hands, then populated it with AI characters. The hybrid is the sweet spot.
After months of fighting the model to produce convincing clay environments, I gave up and just... made the set out of real clay on my kitchen table. Then composited AI-animated characters into it. This hybrid approach solved more problems than any pure-AI technique.
Why real set + AI characters works so well:
The real clay set brings GENUINE texture, real thumbprints, real imperfect lighting interaction that the model can never fully fake. The world is authentically handmade because it IS.
The AI handles the CHARACTERS — the hard part, the performance, the squash-and-stretch animation that would take me weeks to do in real stop-motion frame by frame.
The character inherits the set's lighting, so compositing is believable, and crucially the character picks up the set's REAL texture language so it reads as belonging to the same handmade world.
The trick to selling it: match the AI character's frame rate to real stop-motion (on twos), add contact shadows where they touch the real set, and run a unified grain over the whole composite so the real and the AI share one 'film stock'. The grain marries them.
It's the best of both: real-world tactility, AI-powered performance. Photos of the real set + the final composite in comments. My kitchen is covered in plasticine and I regret nothing.