---
title: "Multi-head attention"
description: "One attention head learns one kind of relationship; running many in parallel — each with its own smaller Q/K/V projections, concatenated back together — le"
canonical_url: "https://fanout.sh/ai-engineering/lessons/multihead-attention"
md_url: "https://fanout.sh/ai-engineering/lessons/multihead-attention.md"
access: "public"
---

# Multi-head attention

One attention head learns one kind of relationship; running many in parallel — each with its own smaller Q/K/V projections, concatenated back together — le

## Public overview

One attention head learns one kind of relationship; running many in parallel — each with its own smaller Q/K/V projections, concatenated back together — lets the layer track syntax, coreference, position, and more simultaneously. Plus the modern efficiency variants (MQA/GQA) that shrink the KV cache by sharing keys and values across heads.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md