---
title: "FlashAttention"
description: "FlashAttention keeps exact attention but changes the memory traffic pattern. It is a clean example of why practical model speed often comes from respecting hardware limits rather than changing the math."
canonical_url: "https://fanout.sh/daily/2026-07-18-flashattention"
md_url: "https://fanout.sh/daily/2026-07-18-flashattention.md"
last_updated: "2026-07-18"
access: "public"
---

# FlashAttention

FlashAttention keeps exact attention but changes the memory traffic pattern. It is a clean example of why practical model speed often comes from respecting hardware limits rather than changing the math.

## Paper details

- Authors: Dao et al.

- Venue: NeurIPS 2022

- Track: Inference

- Difficulty: Hard

- Reading time: 55 min

- Original paper: https://arxiv.org/abs/2205.14135

## What you will learn

- Why attention can be IO-bound even when the formula looks simple.

- How tiling and recomputation reduce high-bandwidth memory reads and writes.

- Why kernel-level details matter for serving long-context models.

## Continue exploring

[Explore Flash Attention](https://fanout.sh/knowledge-graph?node=ai-research%3Aconcept%3Aflash-attention): Jump into Fanout's efficient-attention topic and continue toward KV cache, batching, and inference performance.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md