Rotary Positional Embedding: [paper]

Let
We need the transformation functions above ($f_q$ and $f_k$), such that they satisfy the below equation, the resultant function should encode relative position information

Also See