Positional embeddings

Attention is order-blind — 'dog bites man' and 'man bites dog' look identical without help — so transformers inject position information into token representations. From learned absolute positions to sinusoidal encodings to RoPE (rotary embeddings), which is what modern open models actually use and what makes long-context extension tricks possible.