Watch context turn into inference memory

The KV cache stores one key and one value for every transformer layer, cached token, and active sequence so autoregressive decoding can reuse earlier attention state.

The deterministic estimate is 2 × layers × KV heads × head dimension × cached tokens × active sequences × bytes per element.

Grouped-query attention lowers cache pressure by sharing key and value heads, while longer contexts and more active sequences multiply memory use.

The calculator isolates KV-cache memory. Model weights, activations, allocator fragmentation, CUDA graphs, and runtime overhead still need separate GPU headroom.