---
title: "DeepSeek V3 From Scratch"
description: "How DeepSeek-V3's MLA keeps a compact record per past token instead of full keys and values, and why moving one multiplication gives the same scores."
canonical_url: "https://fanout.sh/ai/lessons/llm-from-scratch-deepseek-v3"
md_url: "https://fanout.sh/ai/lessons/llm-from-scratch-deepseek-v3.md"
access: "pro-preview"
---

# DeepSeek V3 From Scratch

How DeepSeek-V3's MLA keeps a compact record per past token instead of full keys and values, and why moving one multiplication gives the same scores.

- Access: Fanout Pro (public preview)
- Module: LLM From Scratch

## A glimpse of DeepSeek V3 From Scratch

When a language model generates another token, it reads information from earlier tokens. Recomputing all that information would waste work, so the model keeps a key-value cache: saved numbers used by attention.

With many attention heads and a long conversation, those saved numbers add up. DeepSeek-V3 uses Multi-head Latent Attention, or MLA, to keep a compact shared record for each past token.

## Continue learning

Pro unlocks guided learning, complete notes, practical walkthroughs, exercises, and every AI Research lesson.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md