Attention

The core mechanism: every token builds a query, key, and value vector; attention scores are scaled dot products between queries and keys, softmaxed into weights that mix the values. That one formula — softmax(QKᵀ/√d)V — is what lets a token pull in exactly the context it needs.