---
title: "LLM serving latency decomposition"
description: "Separates a request's waiting, prompt-processing, and iterative decode contributions so bottlenecks are not hidden in one average."
canonical_url: "https://fanout.sh/labs/math-decoder/formula/llm-serving-latency-decomposition"
md_url: "https://fanout.sh/labs/math-decoder/formula/llm-serving-latency-decomposition.md"
last_updated: "2026-08-09"
access: "public"
---

# LLM serving latency decomposition

Separates a request's waiting, prompt-processing, and iterative decode contributions so bottlenecks are not hidden in one average.

## Public overview

Request latency is approximately queueing time plus prefill time plus output token count minus one times time per output token.

Total response time is waiting plus prompt work plus the cadence cost of every generated token after the first.

Addition composes sequential critical-path phases, while multiplication expands the repeated decode-token phase.

Continuous batching, preemption, variable TPOT, speculative decoding, and overlap can make real request traces depart from this simple additive model.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md