---
title: "Superposition Yields Robust Neural Scaling"
description: "Strong superposition changes representation error from the cost of ignored features to interference among shared directions. Follow the controlled toy model, the one-over-width overlap geometry, the four-family LLM fit, and the causal boundary the matching exponents do not cross."
canonical_url: "https://fanout.sh/daily/2026-11-20-superposition-neural-scaling"
md_url: "https://fanout.sh/daily/2026-11-20-superposition-neural-scaling.md"
last_updated: "2026-11-20"
access: "public"
---

# Superposition Yields Robust Neural Scaling

Strong superposition changes representation error from the cost of ignored features to interference among shared directions. Follow the controlled toy model, the one-over-width overlap geometry, the four-family LLM fit, and the causal boundary the matching exponents do not cross.

## Paper details

- Authors: Yizhou Liu, Ziming Liu, and Jeff Gore

- Venue: NeurIPS 2025 Best Paper runner-up

- Track: AI Research

- Difficulty: Hard

- Reading time: 11 min

- Original paper: https://arxiv.org/abs/2505.10465v4

## What you will learn

- How decay and row growth move the same sparse autoencoder between weak and strong superposition.

- Why isotropic feature directions have mean squared overlap 1/m, giving interference a width exponent near one.

- What the OPT, GPT-2, Qwen, Pythia, and Chinchilla comparisons support, and why they do not isolate superposition as the cause of total LLM scaling.

## Continue exploring

[Build the superposition prerequisite](https://fanout.sh/daily/2026-10-10-toy-models-superposition): See how sparse features first learn to share non-orthogonal directions before asking how their interference changes with width.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md
