Sigmoid Loss for Language Image Pre-Training

SigLIP replaces CLIP's two batch-wide softmax choices with an independent sigmoid decision for every image-text pair. Follow the signed pair loss, the learned bias, the chunked distributed implementation, and the batch-size evidence that makes 32,000 more important than one million.

Reading focus: How a similarity matrix becomes independent positive and negative pair decisions instead of two globally normalized classification problems. Why a learned bias matters when every row contains one positive and thousands of negative image-text pairs. What the memory-fit and batch-size sweeps establish, and why false negatives, optimizer instability, and private data still bound the result.

ICCV 2023 Oral. Xiaohua Zhai et al.. 12 min read, medium difficulty.