Dot Products, Softmax & Attention Math
Build the attention formula by hand from three moves: score pairs of vectors with dot products, turn scores into weights with softmax, then sum.
Build the attention formula by hand from three moves: score pairs of vectors with dot products, turn scores into weights with softmax, then sum.