---
title: "Kullback-Leibler divergence"
description: "Measures the expected log-density mismatch when outcomes follow p but are represented using q."
canonical_url: "https://fanout.sh/labs/math-decoder/formula/kullback-leibler-divergence"
md_url: "https://fanout.sh/labs/math-decoder/formula/kullback-leibler-divergence.md"
last_updated: "2026-08-09"
access: "public"
---

# Kullback-Leibler divergence

Measures the expected log-density mismatch when outcomes follow p but are represented using q.

## Public overview

K L divergence from p to q equals the sum over x of p of x times the logarithm of p of x divided by q of x.

Ask how surprised you would be, on average under p, if you used q's probabilities instead.

The p-weighted summation is an expectation: average the log probability ratio under outcomes drawn from p.

KL is a divergence rather than a metric: reversing p and q generally changes the value, and support mismatch can make it infinite.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md