---
title: "Entropy & Information Theory"
description: "Entropy puts a number on uncertainty. Measure how surprising one outcome is in bits, average that surprise over a distribution, then read the formula."
canonical_url: "https://fanout.sh/ai/lessons/math-fundamentals-entropy"
md_url: "https://fanout.sh/ai/lessons/math-fundamentals-entropy.md"
access: "pro-preview"
---

# Entropy & Information Theory

Entropy puts a number on uncertainty. Measure how surprising one outcome is in bits, average that surprise over a distribution, then read the formula.

- Access: Fanout Pro (public preview)
- Module: Math Fundamentals

## A glimpse of Entropy & Information Theory

A coin that always lands heads is easy to predict. A fair coin leaves you unsure. Entropy puts a number on that uncertainty, using the probabilities you assign to the outcomes.

Start with a simpler question: how surprising is one outcome? An outcome with probability 1/2 carries one bit of surprise. Probability 1/4 carries two bits; probability 1/8 carries three. Each halving adds a bit.

## Continue learning

Pro unlocks guided learning, complete notes, practical walkthroughs, exercises, and every AI Research lesson.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md