---
title: "Sparse attention"
description: "Full attention costs O(n²) in sequence length, so long-context models attend selectively: sliding windows, dilated patterns, or learned sparsity. What surv"
canonical_url: "https://fanout.sh/ai-engineering/lessons/sparse-attention"
md_url: "https://fanout.sh/ai-engineering/lessons/sparse-attention.md"
access: "public"
---

# Sparse attention

Full attention costs O(n²) in sequence length, so long-context models attend selectively: sliding windows, dilated patterns, or learned sparsity. What surv

## Public overview

Full attention costs O(n²) in sequence length, so long-context models attend selectively: sliding windows, dilated patterns, or learned sparsity. What survives at 128k+ contexts, and the long-range-reference problem sparse patterns must not break (the pronoun that points 1000 words back).

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md