DINOv2: Learning Robust Visual Features without Supervision
DINOv2 trains one visual backbone to preserve both whole-image meaning and patch-level structure without captions or class labels. Follow LVD-142M curation, the moving teacher, masked patch targets, feature spreading, frozen-transfer evidence, and the data and bias boundaries around general-purpose features.
Reading focus: How visually retrieved web images turn a 1.2-billion-image pool into the curated LVD-142M pretraining set. Why crop-level self-distillation, masked patch prediction, and KoLeo feature spreading protect different uses of one representation. What frozen linear probes, dense prediction heads, distillation ablations, and the Dollar Street audit establish, and what private data still prevents reproducing.
TMLR 2024. Maxime Oquab et al.. 15 min read, medium difficulty.