---
title: "DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving"
description: "Prefill is compute-bound and decoding is bandwidth-bound, so sharing a GPU makes each wait on the other. Follow DistServe's split into prefill and decode pools, the per-GPU goodput arithmetic, the KV-cache transfer it adds, and what its evaluation does and does not show."
canonical_url: "https://fanout.sh/daily/2026-11-09-distserve-prefill-decode-disaggregation"
md_url: "https://fanout.sh/daily/2026-11-09-distserve-prefill-decode-disaggregation.md"
last_updated: "2026-11-09"
access: "public"
---

# DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving

Prefill is compute-bound and decoding is bandwidth-bound, so sharing a GPU makes each wait on the other. Follow DistServe's split into prefill and decode pools, the per-GPU goodput arithmetic, the KV-cache transfer it adds, and what its evaluation does and does not show.

## Paper details

- Authors: Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, and Hao Zhang

- Venue: OSDI 2024

- Track: Inference

- Difficulty: Medium

- Reading time: 16 min

- Original paper: https://arxiv.org/abs/2401.09670v3

## What you will learn

- Why time to first token and time per output token pull a shared GPU in opposite directions, and how goodput scores both.

- How separate prefill and decode pools turn capacity planning into a ratio between two phase rates, with a KV-cache handoff between them.

- What the placement search, the transfer measurements, and the vLLM++ ablation establish, and where synthetic arrivals and author-set deadlines bound the claim.

## Continue exploring

[Size a split with numbers](https://fanout.sh/blog/disaggregated-prefill-and-decode): Continue into Fanout's practical guide to sizing prefill and decode pools, cache transfer, and when a split pays.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md
