Attention weight matrix mathematical notation

Attention weight matrix is a recurring research-paper notation family. Normalizes attention scores along the key axis so each query row supplies weights for a value-vector mixture.

Attention weight matrix: Normalizes attention scores along the key axis so each query row supplies weights for a value-vector mixture. Example: Every query distributes one unit of weight across visible key positions.