Rotary position embedding mathematical notation

Rotary position embedding is a recurring research-paper notation family. Rotates pairs of query and key features by position-dependent angles so their dot product carries relative-position information.

Rotary position embedding: Rotates pairs of query and key features by position-dependent angles so their dot product carries relative-position information. Example: Rotate the query by an angle determined by its token position.