Are Emergent Abilities of Large Language Models a…

This paper asks whether sudden capability jumps in language-model scaling curves belong to the models or to the metrics used to score them. Read it to see how exact match can hide smooth partial progress, why fixed outputs look different under Token Edit Distance and Brier Score, and what evidence a serious emergence claim should survive.

Reading focus: How exact-match accuracy can transform smooth token-level improvement into a flat-then-jump curve. Why re-scoring fixed outputs isolates the metric from the model, prompt set, and training run. Which claims survive the paper's BIG-Bench meta-analysis, induced vision experiments, and stated limitations.

NeurIPS 2023 Outstanding Paper. Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. 34 min read, medium difficulty.