---
title: "Trace the shapes through a transformer"
description: "Trace four tokens through a small decoder. Check matrix shapes and find mistakes involving position vectors, token mixing, and the two softmax operations."
canonical_url: "https://fanout.sh/ai-engineering/lessons/summary-pdf--transformer-architecture"
md_url: "https://fanout.sh/ai-engineering/lessons/summary-pdf--transformer-architecture.md"
access: "public"
---

# Trace the shapes through a transformer

Trace four tokens through a small decoder. Check matrix shapes and find mistakes involving position vectors, token mixing, and the two softmax operations.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md