Multi-query attention mathematical notation

Multi-query attention is a recurring research-paper notation family. Uses many query heads but shares one key head and one value head across all of them to reduce cache size and memory traffic.

Multi-query attention: Uses many query heads but shares one key head and one value head across all of them to reduce cache size and memory traffic. Example: Reuse the same cached keys and values for every query head.