---
title: "Autoregressive sequence log-likelihood"
description: "Decomposes sequence likelihood into next-token prediction terms that an autoregressive model can train and evaluate directly."
canonical_url: "https://fanout.sh/labs/math-decoder/formula/autoregressive-sequence-log-likelihood"
md_url: "https://fanout.sh/labs/math-decoder/formula/autoregressive-sequence-log-likelihood.md"
last_updated: "2026-08-09"
access: "public"
---

# Autoregressive sequence log-likelihood

Decomposes sequence likelihood into next-token prediction terms that an autoregressive model can train and evaluate directly.

## Public overview

Log probability of the sequence x one through T equals the sum from t equals one through T of the log probability of token x t given all earlier tokens.

Score a whole sequence by adding how well the model predicted every observed token from its earlier context.

The summation is the outer reduction; logarithms convert a joint probability product into additive token evidence.

This is the training and evaluation factorization behind next-token prediction; causal masking prevents a term from accessing future tokens.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md