---
title: "Positional embeddings"
description: "Attention is order-blind — 'dog bites man' and 'man bites dog' look identical without help — so transformers inject position information into token represe"
canonical_url: "https://fanout.sh/ai-engineering/lessons/positional-embeddings"
md_url: "https://fanout.sh/ai-engineering/lessons/positional-embeddings.md"
access: "public"
---

# Positional embeddings

Attention is order-blind — 'dog bites man' and 'man bites dog' look identical without help — so transformers inject position information into token represe

## Public overview

Attention is order-blind — 'dog bites man' and 'man bites dog' look identical without help — so transformers inject position information into token representations. From learned absolute positions to sinusoidal encodings to RoPE (rotary embeddings), which is what modern open models actually use and what makes long-context extension tricks possible.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md