From Tensor Buffer to Distributed Memory Hierarchy

This survey treats the KV cache as a distributed memory-management problem, not just a temporary tensor. It is a useful July 20 pick for understanding why LLM serving depends on locality, lifetime, ownership, substrate, and the tradeoffs behind paged, shared, disaggregated, and tiered cache designs.

Reading focus: Why the KV cache becomes a first-order serving object as context and concurrency grow. How locality, lifetime, ownership, and substrate separate different KV-cache architectures. Which measurements are still missing when teams evaluate LLM serving systems.

arXiv 2026. Li, Wang, and Chen. 50 min read, medium difficulty.