---
title: "KV Cache"
description: "During generation, every new token would naively recompute attention keys and values for the whole history. The KV cache stores them once and reuses them,"
canonical_url: "https://fanout.sh/ai-engineering/lessons/kv-cache"
md_url: "https://fanout.sh/ai-engineering/lessons/kv-cache.md"
access: "public"
---

# KV Cache

During generation, every new token would naively recompute attention keys and values for the whole history. The KV cache stores them once and reuses them,

## Public overview

During generation, every new token would naively recompute attention keys and values for the whole history. The KV cache stores them once and reuses them, turning quadratic regeneration into linear appends — the single most important inference optimization, and the reason long chats eat GPU memory.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md