App UI demo: I finally got a phone screen to scroll BELIEVABLY in Kling, here's the trick

by @marcustorres_studio35 in c/product-demos · · App / UI Demo

App demo videos are their own special hell. You need a phone, in a hand or on a desk, with the screen showing your actual app, and the screen content has to MOVE correctly, scroll, tap, transition, without the UI melting into AI-interface-soup (buttons that aren't buttons, text that's greeked nonsense, icons that morph).

What I do now: I DON'T let the AI generate the screen content. I generate the phone and the hand and the environment with a BLANK or green screen, then I composite my real app recording (an actual screen capture from the real app) onto the screen with corner-pin tracking. The AI makes the beautiful physical world, the real app provides the real, accurate, legible UI.

This is the honest way AND the only way that produces legible UI, because no model can render your actual app's interface correctly, it'll invent a plausible-looking fake that's useless for a demo. Real screen recording, AI everything-else, corner-pin to marry them. Demo in comments, the scroll is buttery and every pixel of the UI is real.

528 points · 8 comments

Comments

  • @ethanpatellab · · 92 points

    compositing a REAL screen recording onto an AI-generated phone is the only correct way to do app demos and it's wild how many people try to make the model render their UI and get illegible greeked garbage, the model CANNOT know your app, stop asking it to

  • @elenamartinez_films287 · · 61 points

    AI-rendered UI is always 'plausible interface-shaped noise,' buttons that don't align, text that's wingdings, it looks like an app from a dream, useless for an actual demo where the UI is the POINT

  • @jameschen_creates · · 48 points

    corner-pin tracking onto an AI phone is the move, how stable is the track when the phone moves? AI footage doesn't have real tracking markers, I'd expect the screen comp to swim

  • @chentanaka · · 41 points

    the trick is to keep the phone motion SLOW and simple (a gentle push-in, a slight tilt), fast or complex phone motion and yeah the track swims because there's no real parallax, slow moves track clean

  • @zarapatel · · 38 points

    this is also the honest path, your demo shows your REAL app doing REAL things, the only AI part is the pretty hand and lighting, zero capability-lying, mod-approved energy