---
title: "Quantization"
description: "Shrink the model itself: store weights in 8-bit or 4-bit instead of 16-bit floats. Post-training quantization is nearly free to apply and surprisingly robu"
canonical_url: "https://fanout.sh/ai-engineering/lessons/quantization"
md_url: "https://fanout.sh/ai-engineering/lessons/quantization.md"
access: "public"
---

# Quantization

Shrink the model itself: store weights in 8-bit or 4-bit instead of 16-bit floats. Post-training quantization is nearly free to apply and surprisingly robu

## Public overview

Shrink the model itself: store weights in 8-bit or 4-bit instead of 16-bit floats. Post-training quantization is nearly free to apply and surprisingly robust; the trade is a quality haircut that grows as bits shrink. The reason a 70B model runs on a single consumer GPU at all.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md