---
title: "Sigmoid Loss for Language Image Pre-Training"
description: "SigLIP replaces CLIP's two batch-wide softmax choices with an independent sigmoid decision for every image-text pair. Follow the signed pair loss, the learned bias, the chunked distributed implementation, and the batch-size evidence that makes 32,000 more important than one million."
canonical_url: "https://fanout.sh/daily/2026-11-05-sigmoid-loss-language-image-pretraining"
md_url: "https://fanout.sh/daily/2026-11-05-sigmoid-loss-language-image-pretraining.md"
last_updated: "2026-11-05"
access: "public"
---

# Sigmoid Loss for Language Image Pre-Training

SigLIP replaces CLIP's two batch-wide softmax choices with an independent sigmoid decision for every image-text pair. Follow the signed pair loss, the learned bias, the chunked distributed implementation, and the batch-size evidence that makes 32,000 more important than one million.

## Paper details

- Authors: Xiaohua Zhai et al.

- Venue: ICCV 2023 Oral

- Track: AI Research

- Difficulty: Medium

- Reading time: 12 min

- Original paper: https://arxiv.org/abs/2303.15343v4

## What you will learn

- How a similarity matrix becomes independent positive and negative pair decisions instead of two globally normalized classification problems.

- Why a learned bias matters when every row contains one positive and thousands of negative image-text pairs.

- What the memory-fit and batch-size sweeps establish, and why false negatives, optimizer instability, and private data still bound the result.

## Continue exploring

[Compare CLIP's contrastive objective](https://fanout.sh/daily/2026-08-23-clip-natural-language-supervision): Start with the original image-text alignment recipe, then compare its row and column softmax normalizers with SigLIP's independent pair losses.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md