W8A8 vs W4A16 quantization: how to choose
W8A8 vs W4A16 quantization looks like an accuracy comparison, and most of page one treats it that way. In production the accuracy gap is mostly settled; the real decision is which resource your deployment is short on.
The notation says what W8A8 vs W4A16 quantization actually changes. W8A8 stores weights and activations in 8 bits, so matrix multiplies run on 8-bit tensor cores. W4A16 stores only weights in 4 bits and keeps the math in 16-bit.
That single difference drives everything: memory footprint, prefill speed, decode speed, and how the answer flips with batch size. Work through the arithmetic before picking a scheme.
What each scheme actually buys
W8A8 buys compute speed. On an H100, FP8 tensor cores peak at roughly twice the throughput of BF16, so the same matmul can finish in about half the time once weights and activations are both 8-bit.
It also halves weight memory: 2 bytes per parameter becomes 1.
W4A16 buys memory and bandwidth. Weights shrink 4x, but every matmul still runs in 16-bit, so the kernel dequantizes each weight tile on the fly before multiplying. Peak compute throughput does not improve at all.
Hardware sets the menu. Ampere GPUs like the A100 have INT8 tensor cores but no FP8 support, which arrived with Hopper. On older cards, choosing W8A8 means INT8 and the calibration work that comes with it.
AWQ and GPTQ are the common W4A16 recipes; FP8 and INT8 SmoothQuant-style schemes are the common W8A8 ones. Fanout's format comparison covers how each assigns its bits.
Start with the memory fit
Bytes per parameter times parameter count is the first test. Llama-3.1-70B holds 70.6 billion parameters: 141 GB at BF16, 71 GB at 8 bits, 35 GB at 4 bits.
On a single 80 GB H100, BF16 does not fit at all. W8A8 fits with about 9 GB to spare, which activations and cache consume quickly. W4A16 fits with roughly 45 GB left over.
The same arithmetic at 8B scale: 16 GB at BF16, 8 GB at 8 bits, 4 GB at 4 bits. A 24 GB consumer card holds the BF16 model but leaves thin cache room at long contexts.
That slack is serving capacity. The KV cache formula converts spare gigabytes into concurrent sequences at your context length.
If the model does not fit, the decision is over before any speed argument starts. This is why 4-bit dominates single-GPU and local deployment.
Prefill is compute-bound, and W8A8 wins it
Prefill pushes the whole prompt through the model at once. Arithmetic intensity is high, tensor cores saturate, and the limit is FLOPS.
W8A8 attacks exactly that limit by running the matmuls at 8-bit throughput. W4A16 does nothing for it, and its dequantization adds work to kernels that were already compute-bound.
A workload dominated by long prompts and short answers, like RAG or document extraction, leans W8A8 for time to first token. Fanout's prefill and decode guide explains the phase split.
Decode at low batch is bandwidth-bound, and W4A16 wins it
Each decode step reads every active weight byte once to produce one token per sequence. At batch one, the step cannot finish faster than model bytes divided by memory bandwidth.
Run that ceiling on one H100 at 3.35 TB per second. The 70B model at 4 bits is 35 GB, so at most about 95 tokens per second. At 8 bits, 71 GB caps the rate near 47. Half the bytes, double the ceiling.
This is roofline logic: bandwidth-bound kernels speed up in proportion to bytes removed. Fanout's roofline guide formalizes when that regime applies.
Latency-sensitive, low-concurrency serving sits in this regime: quiet chat deployments, on-device assistants, internal tools. That is why W4A16 is the default there.
Batch size flips the answer
Weight bytes are read once per step no matter how many sequences share the pass. At batch 32, per-token weight traffic falls 32x, and the matmuls drift from bandwidth-bound to compute-bound.
W4A16's advantage evaporates in that regime while its dequantization cost stays. The QServe paper measured 20 to 90 percent runtime overhead from INT4 dequantization in large-batch serving.
The largest quantization accuracy study to date reached the same operational split. Give Me BF16 or Give Me Death ran over 500,000 evaluations across the Llama-3.1 family.
Its deployment finding: W4A16 is the most cost-efficient scheme for synchronous, latency-driven serving, while W8A8 dominates asynchronous continuous batching, where the scheduler keeps batches full.
So the same model on the same GPU can favor either scheme depending on traffic. Estimate your typical batch occupancy before choosing.
The crossover is not exotic. A public API under continuous batching routinely runs effective batch sizes in the tens, deep in compute-bound territory. An internal tool with a handful of users may never leave single digits.
Accuracy is a constraint check, not the decider
The 500,000-evaluation study found FP8 W8A8 effectively lossless at every model scale it tested.
It measured only 1 to 3 percent degradation for INT8 W8A8, and found W4A16 more competitive than its reputation suggested.
Verify on your own evals, since sensitivity varies by task and model size. But do not expect accuracy to break the tie between well-implemented schemes at these widths.
The KV cache is a separate budget with its own decision. KV cache quantization shrinks a different set of bytes, and its failure modes have nothing to do with weight format.
W8A8 vs W4A16 quantization, in short
Three tests decide it. Fit: if 8-bit leaves no cache headroom on your GPU, take W4A16. Phase: prefill-heavy leans W8A8, decode-heavy at low batch leans W4A16. Traffic: saturated continuous batching leans W8A8.
- Single GPU, biggest model that fits: W4A16.
- High-QPS API on Hopper or newer: W8A8 in FP8, which is also the safest format for accuracy.
- Long-prompt RAG with short outputs: W8A8 for prefill throughput.
- Interactive latency at low concurrency: W4A16 for the bandwidth ceiling.
- Ampere and older without FP8 support: INT8 W8A8 or W4A16, and benchmark both.
Then benchmark the two finalists on your own traffic. Kernel quality varies by engine and GPU generation, and it moves results more than any published table.