Introduced Packers/Matmul for QC2W weights and QP8 activations - #10877
Open
damdoo001-arm wants to merge 2 commits into
Open
Introduced Packers/Matmul for QC2W weights and QP8 activations#10877damdoo001-arm wants to merge 2 commits into
damdoo001-arm wants to merge 2 commits into
Conversation
Signed-off-by: Damien Dooley <damien.dooley@arm.com>
damdoo001-arm
changed the title
damdoo001-arm
changed the title
qukhan
requested changes
Jul 29, 2026
| ExternalProject_Add(kleidiai | ||
| URL https://gitlab.arm.com/kleidi/kleidiai/-/archive/b87ef9c94f45f11c81a6b1fdaed1b2b45ea58c0c/kleidiai-b87ef9c94f45f11c81a6b1fdaed1b2b45ea58c0c.zip | ||
| URL_HASH SHA256=b147799b94c51f5e57492930bfd9e5294fb7ffe44fee1dbcd3f8048adeedd5e3 | ||
| URL https://github.com/ARM-software/kleidiai/archive/13cd35993d8439143aff1e756a862d366acded0d.zip |
Comment on lines
+273
to
+274
| "//:operator_h", | ||
| "//src/configs:config_hdrs", |
Collaborator
There was a problem hiding this comment.
Can you explain why this change is needed? What do you add that requires this?
Author
There was a problem hiding this comment.
The new QP8/QC2W selection tests directly use XNN_FLAG_INLINE_LHS_PACKING from operator.h and xnn_init_qp8_f32_qc2w_gemm_config() from config.h. The latter checks whether the configuration is available before running. I’ve narrowed config_hdrs to the specific config_h target.
Comment on lines
+300
to
+301
| std::fill(channelwise_zero_point.begin(), channelwise_zero_point.end(), | ||
| 0.0f); |
Collaborator
There was a problem hiding this comment.
The constructor of channelwise_zero_point just above already fills the vector with zeros.
Author
There was a problem hiding this comment.
Agree. I've removed this. Thanks.
| void TestStaticB(xnn_datatype convert_to = xnn_datatype_invalid, | ||
| size_t block_size = no_blockwise) { | ||
| size_t block_size = no_blockwise, | ||
| bool require_qp8_qc2w = false) { |
Collaborator
There was a problem hiding this comment.
Change this bool to an enum. If other configs need a special setup it won't scale well to have to add new booleans everytime.
| }, | ||
| 'third_party/kleidiai/src': { | ||
| 'url': 'https://gitlab.arm.com/kleidi/kleidiai@v1.25.0', | ||
| 'url': 'https://github.com/ARM-software/kleidiai.git@v1.29.0', |
| strip_prefix = "kleidiai-13cd35993d8439143aff1e756a862d366acded0d", | ||
| urls = [ | ||
| "https://gitlab.arm.com/kleidi/kleidiai/-/archive/b87ef9c94f45f11c81a6b1fdaed1b2b45ea58c0c/kleidiai-b87ef9c94f45f11c81a6b1fdaed1b2b45ea58c0c.zip", | ||
| "https://github.com/ARM-software/kleidiai/archive/13cd35993d8439143aff1e756a862d366acded0d.zip", |
Signed-off-by: Damien Dooley <damien.dooley@arm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add an SME2-optimized
QP8/F32/QC2Wfully connected path using KleidiAI 1.29.The integration adds wrappers for two signed INT2 weight kernels:
1x64c4, targeting single-row/decode workloads.16x64c4, targeting multi-row/prefill workloads.The path dynamically quantizes FP32 activations to INT8, uses per-channel signed INT2 weights, and produces FP32 output. It also adds:
Testing
Validated locally:
//test:packing_test//test:qp8_f32_qc2w_gemm_minmax_test//test/operators:fully_connected_nc_test//test/subgraph:fully_connected_testValidated on the Android board:
Dependency
Requires KleidiAI 1.29, which provides the SME2 DOT and MOPA INT2 kernels and the corresponding NxK weight packer.