Grouped-query attention mathematical notation

Grouped-query attention is a recurring research-paper notation family. Partitions query heads into groups whose members share one key and value head, interpolating between multi-head and multi-query attention.

Grouped-query attention: Partitions query heads into groups whose members share one key and value head, interpolating between multi-head and multi-query attention. Example: Assign several query heads to each stored key-value head.