---
title: "Are Emergent Abilities of Large Language Models a Mirage?"
description: "This paper asks whether sudden capability jumps in language-model scaling curves belong to the models or to the metrics used to score them. Read it to see how exact match can hide smooth partial progress, why fixed outputs look different under Token Edit Distance and Brier Score, and what evidence a serious emergence claim should survive."
canonical_url: "https://fanout.sh/daily/2026-10-16-emergent-abilities-mirage"
md_url: "https://fanout.sh/daily/2026-10-16-emergent-abilities-mirage.md"
last_updated: "2026-10-16"
access: "public"
---

# Are Emergent Abilities of Large Language Models a Mirage?

This paper asks whether sudden capability jumps in language-model scaling curves belong to the models or to the metrics used to score them. Read it to see how exact match can hide smooth partial progress, why fixed outputs look different under Token Edit Distance and Brier Score, and what evidence a serious emergence claim should survive.

## Paper details

- Authors: Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo

- Venue: NeurIPS 2023 Outstanding Paper

- Track: AI Research

- Difficulty: Medium

- Reading time: 34 min

- Original paper: https://proceedings.neurips.cc/paper_files/paper/2023/hash/adc98a266f45005c403b8311ca7e8bd7-Abstract-Conference.html

## What you will learn

- How exact-match accuracy can transform smooth token-level improvement into a flat-then-jump curve.

- Why re-scoring fixed outputs isolates the metric from the model, prompt set, and training run.

- Which claims survive the paper's BIG-Bench meta-analysis, induced vision experiments, and stated limitations.

## Continue exploring

[Start with scaling laws](https://fanout.sh/daily/2026-07-17-scaling-laws): Review how performance changes with model scale before asking whether a bend in that curve reflects the model or the measurement.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md