---
title: "RoPE relative-position rotation"
description: "Shows why rotary position embedding makes a query-key attention score depend on relative position even though each vector receives an absolute-position…"
canonical_url: "https://fanout.sh/labs/math-decoder/formula/rotary-position-relative-dot-product"
md_url: "https://fanout.sh/labs/math-decoder/formula/rotary-position-relative-dot-product.md"
last_updated: "2026-08-09"
access: "public"
---

# RoPE relative-position rotation

Shows why rotary position embedding makes a query-key attention score depend on relative position even though each vector receives an absolute-position…

## Public overview

Shows why rotary position embedding makes a query-key attention score depend on relative position even though each vector receives an absolute-position rotation.

Rotated query at position m transpose times rotated key at position n equals query m transpose times the relative rotary matrix at position difference n minus m times key n.

Rotate queries and keys separately, yet make their similarity depend on the distance between token positions.

The dot product is the output operation; orthogonal rotation algebra transforms its two absolute phases into one relative phase.

Implementations apply sine and cosine elementwise rather than constructing full block-diagonal matrices, but the same identity governs the result.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md