---
title: "Transformer Architecture Refresher"
description: "A map of the decoder-only transformer behind Llama 3.1 8B: the embedding, 32 identical blocks, attention, the gated MLP and position, with every shape."
canonical_url: "https://fanout.sh/inference-eng/curriculum/00-06"
md_url: "https://fanout.sh/inference-eng/curriculum/00-06.md"
access: "public"
---

# Transformer Architecture Refresher

A map of the decoder-only transformer behind Llama 3.1 8B: the embedding, 32 identical blocks, attention, the gated MLP and position, with every shape.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md