---
title: "Mixture of experts"
description: "Scale parameters without scaling compute: replace each MLP with many 'expert' MLPs and a router that activates only a couple per token. A trillion-paramete"
canonical_url: "https://fanout.sh/ai-engineering/lessons/mixture-of-experts"
md_url: "https://fanout.sh/ai-engineering/lessons/mixture-of-experts.md"
access: "public"
---

# Mixture of experts

Scale parameters without scaling compute: replace each MLP with many 'expert' MLPs and a router that activates only a couple per token. A trillion-paramete

## Public overview

Scale parameters without scaling compute: replace each MLP with many 'expert' MLPs and a router that activates only a couple per token. A trillion-parameter MoE can run at the cost of a much smaller dense model — the trick behind Mixtral, DeepSeek-V3, and most frontier models today. The catch: routing balance and memory footprint.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md