Scalars, vectors, matrices, and tensors
Classify numerical objects by their axes and the roles those axes play.
Notes
Use shapes as a proofreader
Treat shapes as type signatures: matching axes may contract, while the axes not consumed by an operation predict the result's shape and meaning.
Contract the shared axis
$$Q\in\mathbb{R}^{2\times3},\quad K\in\mathbb{R}^{4\times3},\quad QK^\top\in\mathbb{R}^{2\times4}$$
Two queries and four keys share a three-feature axis; transposing K aligns that axis, removes it through dot products, and leaves a two-by-four score grid.
A shared number is not enough
$$QK$$
Tempting reading: Multiplying Q and K directly because both declarations visibly contain a feature dimension of size three.
Inspect the touching axes
Without transpose, the touching dimensions are 3 and 4, so ordinary matrix multiplication is undefined. K transpose makes them both 3.
Trace attention axes
$$\operatorname{softmax}\left(\frac{QK^\top}{\sqrt{d_k}}\right)V$$
A Transformer paper compresses pairwise scoring and value mixing into the attention formula.
Name what survives
Annotate every axis before calculating: the feature axis d sub k contracts in Q K transpose, while query and key-position axes form the score matrix.