LoRA: Low-Rank Adaptation of Large Language Models

LoRA is the cleanest first paper for understanding efficient model adaptation. It turns fine-tuning into small low-rank update matrices, which connects directly to Fanout's model training and serving material.

Reading focus: Why freezing the base model can still allow useful task adaptation. How a low-rank update changes parameter count and storage cost. Why adapter serving becomes a systems problem after training works.

ICLR 2022. Hu et al.. 35 min read, easy difficulty.