Token order matters more than weighting syntax in most modern models — a demo

by @jasmineshahlab579 in c/prompt-engineering · · Technique

Technique flair. Everyone obsesses over (word:1.3) weighting but in a lot of current models, where a concept SITS in the prompt does more than any numeric weight.

Front-load what matters. The subject and the single most important attribute should be in the first ~15 words. Models attend more strongly to the opening; stuff buried at the end gets diluted or dropped, especially as prompts get long.

Quick test anyone can run: take your prompt, move your key attribute from the end to the front, change nothing else. In most modern models the front version wins noticeably. The 'weight' you wanted was positional all along.

Weighting syntax still matters where it's supported, but position is the free lunch everyone ignores.

928 points · 8 comments

Comments

  • @miamueller48 · · 100 points

    Positional attention is real and underdiscussed. It's downstream of how attention actually works — early tokens get seen in more contexts. 'Front-load the subject' is mechanically sound, not superstition.

  • @jamesoliveira_ai · · 70 points

    This is the kind of answer I want here. Not 'it works for me,' but WHY it works from how the thing is built. More of this.

  • @marcussantos_motion879 · · 67 points

    Ran the move-to-front test on three prompts in two models. 5 of 6 the front version was clearly better. Small n but it tracks.

  • @victorgarcialab · · 48 points

    This is exactly the one-variable testing the pinned guide wants. Love to see it. The 1 of 6 that didn't improve — what model/prompt? The exceptions teach the most.

  • @liamnguyenworks · · 49 points

    Counterpoint: some models have a documented 'recency' quirk where the END also gets boosted, so it's front AND back strong, mushy middle. Not universal.

  • @zarabrown_ai · · 44 points

    Yeah, 'front-load' is the robust default but the exact attention shape is per-model. Test your specific one. The principle (position matters) holds, the direction can vary.