[DRAFT/DO NOT REVIEW] Add Rotary Embedding from ONNX Opset 23 #23507
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| @@ -0,0 +1,139 @@ | |||
| // Copyright (c) Microsoft Corporation. All rights reserved. | |||
Check warning
Code scanning / lintrunner
CLANGFORMAT/format Warning
| @@ -0,0 +1,30 @@ | |||
| // Copyright (c) Microsoft Corporation. All rights reserved. | |||
Check warning
Code scanning / lintrunner
CLANGFORMAT/format Warning
| @@ -0,0 +1,633 @@ | |||
| // Copyright (c) Microsoft Corporation. All rights reserved. | |||
Check warning
Code scanning / lintrunner
CLANGFORMAT/format Warning test
| if (rotary_emb_dim < head_size) { | ||
| std::memcpy(output_data + rotary_emb_dim, | ||
| input_data + rotary_emb_dim, | ||
| (head_size - rotary_emb_dim) * sizeof(T)); |
Check warning
Code scanning / PREfast
Arithmetic overflow: Using operator '-' on a 4 byte value and then casting the result to a 8 byte value. Cast the value to the wider type before calling operator '-' to avoid overflow (io.2).
| if (rotary_emb_dim < head_size) { | ||
| std::memcpy(output_data + rotary_emb_dim, | ||
| input_data + rotary_emb_dim, | ||
| (head_size - rotary_emb_dim) * sizeof(T)); |
Check warning
Code scanning / PREfast
Arithmetic overflow: Using operator '-' on a 4 byte value and then casting the result to a 8 byte value. Cast the value to the wider type before calling operator '-' to avoid overflow (io.2).
| } | ||
|
|
||
| // Interleaved = true, pos ids shape = (1) | ||
| TEST(RotaryEmbeddingONNXTest, RotaryEmbeddingONNX_Interleaved_LargeData_LlamaMSFT) { |
Check warning
Code scanning / CodeQL
Poorly documented large function Warning test
| } | ||
|
|
||
| // Interleaved = false, pos ids shape = (1) | ||
| TEST(RotaryEmbeddingONNXTest, RotaryEmbeddingONNX_NotInterleaved_LargeData_LlamaMSFT) { |
Check warning
Code scanning / CodeQL
Poorly documented large function Warning test
| @@ -0,0 +1,199 @@ | |||
| // Copyright (c) Microsoft Corporation. All rights reserved. | |||
Check warning
Code scanning / lintrunner
CLANGFORMAT/format Warning
Description
Add kernels for Rotary Embedding op that is planned to be added in ONNX opset 23.
Motivation and Context
This PR is currently experimental, and the kernels are added to a temporary location in contrib_ops. Once the following happens:
These kernels will be migrated from contrib_ops/onnx_std_exp to the correct location.