Segment Anything

Segment Anything asks whether one vision model can turn a point, box, or rough mask into the pixels of a reasonable object, even on unfamiliar images. Read it to see how one reusable image embedding, a lightweight prompt decoder, ambiguity-aware mask candidates, and a model-in-the-loop data engine make segmentation feel like an interface rather than a fixed label set.

Reading focus: How promptable segmentation replaces a fixed class vocabulary with spatial instructions while allowing several answers to be valid for one ambiguous point. Why SAM computes the expensive image embedding once, then combines cheap prompt embeddings with a lightweight mask decoder for repeated interaction. How the assisted, semi-automatic, and fully automatic data-engine stages produced SA-1B, and where fine structures, heavy encoding, text prompts, data coverage, and specialist tools still limit the result.

ICCV 2023. Kirillov et al.. 35 min read, easy difficulty.