---
title: "From Tensor Buffer to Distributed Memory Hierarchy"
description: "This survey treats the KV cache as a distributed memory-management problem, not just a temporary tensor. It is a useful July 20 pick for understanding why LLM serving depends on locality, lifetime, ownership, substrate, and the tradeoffs behind paged, shared, disaggregated, and tiered cache designs."
canonical_url: "https://fanout.sh/daily/2026-07-20-kv-cache-management-survey"
md_url: "https://fanout.sh/daily/2026-07-20-kv-cache-management-survey.md"
last_updated: "2026-07-20"
access: "public"
---

# From Tensor Buffer to Distributed Memory Hierarchy

This survey treats the KV cache as a distributed memory-management problem, not just a temporary tensor. It is a useful July 20 pick for understanding why LLM serving depends on locality, lifetime, ownership, substrate, and the tradeoffs behind paged, shared, disaggregated, and tiered cache designs.

## Paper details

- Authors: Li, Wang, and Chen

- Venue: arXiv 2026

- Track: Inference

- Difficulty: Medium

- Reading time: 50 min

- Original paper: https://arxiv.org/abs/2607.02574

## What you will learn

- Why the KV cache becomes a first-order serving object as context and concurrency grow.

- How locality, lifetime, ownership, and substrate separate different KV-cache architectures.

- Which measurements are still missing when teams evaluate LLM serving systems.

## Continue exploring

[Explore KV Cache](https://fanout.sh/knowledge-graph?node=ai-research%3Aconcept%3Akv-cache): Open Fanout's KV-cache concept node and connect the survey to attention, batching, PagedAttention, prefix caching, and inference serving.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md