TTFT, TPOT, ITL, E2E & Throughput
The LLM latency and throughput metrics, from time to first token and inter-token latency to end-to-end latency and tokens per second, on a real timeline.
The LLM latency and throughput metrics, from time to first token and inter-token latency to end-to-end latency and tokens per second, on a real timeline.