QLoRA: Efficient Finetuning of Quantized LLMs

QLoRA made large-model fine-tuning fit radically smaller memory budgets by storing a frozen base in 4-bit NormalFloat, dequantizing blocks for 16-bit computation, and training only LoRA adapters. Follow the precision boundary, memory devices, ablations, and qualified chatbot evidence.

Reading focus: Why four-bit storage does not mean four-bit matrix multiplication, gradients, activations, or optimizer state. How NF4, double quantization, paged optimizers, and broad LoRA coverage each remove a different memory or accuracy bottleneck. What the MMLU datatype comparisons establish and why the 99.3 percent Guanaco headline is bounded by its prompts, judge, and confidence interval.

NeurIPS 2023. Tim Dettmers et al.. 13 min read, hard difficulty.