---
title: "Watch context turn into inference memory"
description: "Estimate KV-cache memory from model shape, context length, batch size, and precision, then compare it with GPU capacity."
canonical_url: "https://fanout.sh/labs/kv-cache"
md_url: "https://fanout.sh/labs/kv-cache.md"
last_updated: "2026-08-06"
access: "public"
---

# Watch context turn into inference memory

Estimate KV-cache memory from model shape, context length, batch size, and precision, then compare it with GPU capacity.

## Public overview

The KV cache stores one key and one value for every transformer layer, cached token, and active sequence so autoregressive decoding can reuse earlier attention state.

The deterministic estimate is 2 × layers × KV heads × head dimension × cached tokens × active sequences × bytes per element.

Grouped-query attention lowers cache pressure by sharing key and value heads, while longer contexts and more active sequences multiply memory use.

The calculator isolates KV-cache memory. Model weights, activations, allocator fragmentation, CUDA graphs, and runtime overhead still need separate GPU headroom.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md