---
title: "Attention"
description: "The core mechanism: every token builds a query, key, and value vector; attention scores are scaled dot products between queries and keys, softmaxed into we"
canonical_url: "https://fanout.sh/ai-engineering/lessons/attention"
md_url: "https://fanout.sh/ai-engineering/lessons/attention.md"
access: "public"
---

# Attention

The core mechanism: every token builds a query, key, and value vector; attention scores are scaled dot products between queries and keys, softmaxed into we

## Public overview

The core mechanism: every token builds a query, key, and value vector; attention scores are scaled dot products between queries and keys, softmaxed into weights that mix the values. That one formula — softmax(QKᵀ/√d)V — is what lets a token pull in exactly the context it needs.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md