---
title: "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale"
description: "Vision Transformer asks how much image-specific structure a model needs when pre-training data becomes very large. Read it to see how a 224-pixel image becomes 196 patch tokens, which convolutional priors disappear, why ResNets lead at smaller scale, where ViT crosses over, and how patch size turns visual detail into a quadratic token-pair bill."
canonical_url: "https://fanout.sh/daily/2026-09-27-vision-transformer-image-patches"
md_url: "https://fanout.sh/daily/2026-09-27-vision-transformer-image-patches.md"
last_updated: "2026-09-27"
access: "public"
---

# An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Vision Transformer asks how much image-specific structure a model needs when pre-training data becomes very large. Read it to see how a 224-pixel image becomes 196 patch tokens, which convolutional priors disappear, why ResNets lead at smaller scale, where ViT crosses over, and how patch size turns visual detail into a quadratic token-pair bill.

## Paper details

- Authors: Alexey Dosovitskiy et al.

- Venue: ICLR 2021

- Track: AI Research

- Difficulty: Easy

- Reading time: 30 min

- Original paper: https://openreview.net/forum?id=YicbFdNTTy

## What you will learn

- How non-overlapping image patches, a shared projection, learned positions, and one class token create a Transformer sequence.

- Why removing locality and translation equivariance hurts at smaller data scales but can pay off after large-scale pre-training.

- How to read the data crossover, compute comparison, attention analysis, and private JFT dependence without turning them into a universal no-convolution claim.

## Continue exploring

[Explore Vision Transformer in Fanout](https://fanout.sh/knowledge-graph?node=ai-research%3Aconcept%3Avision-transformer-vit): Connect patch tokens to self-attention, position embeddings, transfer learning, image resolution, and the vision models that followed.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md