Byte Latent Transformer: Patches Scale Better Than Tokens

BLT replaces a fixed subword vocabulary with entropy-driven byte patches. Follow the local encoder, global patch Transformer, fixed-inference scaling trade, character robustness gains, test-time threshold caveat, and the wall-clock evidence still missing.

Reading focus: How next-byte entropy creates variable-length patches without assigning a learned token identity to each span. Why pooling bytes into fewer global positions can trade sequence work for a larger Transformer at fixed theoretical inference compute. What the matched-scale, fixed-inference, character, and noisy-text results establish, and where threshold changes and wall-clock costs narrow the claim.

ACL 2025. Artidoro Pagnoni et al.. 12 min read, hard difficulty.