DeepSeekMath: Pushing the Limits of Mathematical…

DeepSeekMath shows how targeted data and reinforcement learning can turn a compact open model into a strong mathematical reasoner. Read it for the data-selection pipeline behind its 120 billion math tokens and for GRPO, a PPO variant that removes the separate critic model by learning from relative rewards within groups of sampled answers.

Reading focus: How an iterative Common Crawl filtering pipeline expands a small high-quality math seed into a 120-billion-token training corpus. How GRPO estimates each answer's advantage by comparing its reward with the other answers sampled for the same problem. Why removing the learned critic reduces memory use while retaining PPO-style clipped policy updates for reinforcement learning.

arXiv 2024. Shao et al.. 55 min read, hard difficulty.