---
title: "KL Divergence"
description: "Score a model's prediction Q against the true distribution P in bits, compute the extra average cost, and see why KL divergence changes with direction."
canonical_url: "https://fanout.sh/ai/lessons/math-fundamentals-kl-divergence"
md_url: "https://fanout.sh/ai/lessons/math-fundamentals-kl-divergence.md"
access: "pro-preview"
---

# KL Divergence

Score a model's prediction Q against the true distribution P in bits, compute the extra average cost, and see why KL divergence changes with direction.

- Access: Fanout Pro (public preview)
- Module: Math Fundamentals

## A glimpse of KL Divergence

Suppose an outcome is red 75% of the time and blue 25% of the time. Call that distribution P = [0.75, 0.25]. A model predicts Q = [0.5, 0.5]. How much does using Q cost us, compared with predicting the actual probabilities?

We need a scoring rule. Give a prediction a cost of −log₂(q) when an outcome assigned probability q happens. Predicting probability 1/2 costs one bit; predicting probability 1/4 costs two bits.

## Continue learning

Pro unlocks guided learning, complete notes, practical walkthroughs, exercises, and every AI Research lesson.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md