---
title: "FlashAttention"
description: "Attention isn't compute-bound — it's memory-bandwidth-bound. FlashAttention restructures the computation into tiles that live in fast on-chip SRAM, never m"
canonical_url: "https://fanout.sh/ai-engineering/lessons/flash-attention"
md_url: "https://fanout.sh/ai-engineering/lessons/flash-attention.md"
access: "public"
---

# FlashAttention

Attention isn't compute-bound — it's memory-bandwidth-bound. FlashAttention restructures the computation into tiles that live in fast on-chip SRAM, never m

## Public overview

Attention isn't compute-bound — it's memory-bandwidth-bound. FlashAttention restructures the computation into tiles that live in fast on-chip SRAM, never materializing the full attention matrix in slow HBM. Exact same math, 2-4× faster and linear memory, which is why it's in every serious training and inference stack.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md