---
title: "Scaled dot-product attention"
description: "Turns query-key similarity scores into normalized weights and uses them to combine value vectors."
canonical_url: "https://fanout.sh/labs/math-decoder/formula/scaled-dot-product-attention"
md_url: "https://fanout.sh/labs/math-decoder/formula/scaled-dot-product-attention.md"
last_updated: "2026-08-09"
access: "public"
---

# Scaled dot-product attention

Turns query-key similarity scores into normalized weights and uses them to combine value vectors.

## Public overview

Attention of Q, K, and V equals softmax of Q times K transpose divided by the square root of d sub k, then multiplied by V.

Each query decides how much attention to place on every key, then blends the associated values using those weights.

The outer data flow is a normalized weighted sum: score, scale, normalize, then mix values.

This is the Transformer paper's core attention primitive; masking and batching add structure around it without changing this central operation.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md