BERT: Pre-training of Deep Bidirectional…

BERT is the pretrain-then-fine-tune paper that made bidirectional Transformer encoders the default for many language understanding tasks. It is useful for separating encoder intuition from decoder-only LLM intuition.

Reading focus: How masked language modeling differs from left-to-right prediction. Why pretraining creates reusable representations for downstream tasks. Where encoder-only Transformers still matter in practical NLP systems.

NAACL 2019. Devlin et al.. 45 min read, medium difficulty.