---
title: "Tiled Attention & Online Softmax"
description: "Why attention seems to need a whole row of scores, and how a running maximum and a running normalizer let softmax work one tile at a time."
canonical_url: "https://fanout.sh/inference-eng/curriculum/14-02"
md_url: "https://fanout.sh/inference-eng/curriculum/14-02.md"
access: "pro-preview"
---

# Tiled Attention & Online Softmax

Why attention seems to need a whole row of scores, and how a running maximum and a running normalizer let softmax work one tile at a time.

- Access: Fanout Pro (public preview)

## Public preview

The last lesson closed on a claim it did not prove: the traffic it had just counted is optional. Proving it takes one observation about softmax. The function subtracts the largest score in a row before exponentiating, a largest score is a fact about the whole row, and knowing a fact about the whole row means having already seen all…

## In this lesson

- Why the row has to exist
- A running maximum and a running normalizer
- Watching the correction fire
- From a row to a tile
- Running it

## Continue learning

Pro unlocks the complete lesson, video, practice, and code where available.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md