My full ComfyUI pose-to-video pipeline for dance, with the node graph and why each piece is there
Wrote this up because the 'just use Viggle' answers in the help threads are leaving beginners stranded the second they want control.
The graph, left to right:
Load driver video → DWPose preprocessor (better than OpenPose for hands/feet, which matter for dance)
Character image → IPAdapter for identity lock
AnimateDiff motion module (v3) as the base animator
ControlNet (pose) fed the DWPose output, strength 0.8 ramping to 0.6 on fast sections
Context options set to 16-frame windows with 4-frame overlap — this is what kills the flicker between batches
Upscale + RIFE frame interpolation to 48fps for the smooth slow-mo look
The settings that actually matter:
DWPose over OpenPose, always, for dance. The foot keypoints are night and day.
Ramp your controlnet strength DOWN on fast motion or you get the stutter-lock where it snaps to each pose like a flipbook.
4-frame context overlap minimum. 2 flickers, 6 gets mushy.
Will answer setup questions below. This is free, go make stuff.