Have you tried guiding an image model with a marked-up map or sketch?
Some image models can follow a simple map or sketch with markers for where things should go, which is a very different kind of prompt from words alone.
Have you tried guiding layout this way, and how closely did the model follow your markers?