Attention output projection mathematical notation
Attention output projection is a recurring research-paper notation family. Maps concatenated or combined per-head attention features back into the model's hidden feature space.
Attention output projection: Maps concatenated or combined per-head attention features back into the model's hidden feature space. Example: Mix information from all attention heads into the residual-stream width.