Find Saturation, SLO Failure & OOM Boundaries
Load test an LLM server to find three boundaries: the rate where it saturates, the rate where the SLO fails, and the context length where memory runs out.
Load test an LLM server to find three boundaries: the rate where it saturates, the rate where the SLO fails, and the context length where memory runs out.