Attention Is All You Need

The Transformer paper replaced recurrence with attention and made sequence modeling easier to parallelize. Read it for the architectural move that turned token-to-token routing into the core primitive behind modern language models.

Reading focus: Why self-attention gives every token a learned view of the full context. How multi-head attention, residual paths, and position encodings fit together. What changed operationally when training became more parallel.

NeurIPS 2017. Vaswani et al.. 45 min read, medium difficulty.