Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(WIP) bitnet and t-mac #23540

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

(WIP) bitnet and t-mac #23540

wants to merge 3 commits into from

Conversation

liqunfu
Copy link
Contributor

@liqunfu liqunfu commented Jan 30, 2025 •
edited
Loading

Preparation for 2bit T-MAC and ternary bit BitNet implementation.

  1. 2bit T-MAC implementation to be added at onnxruntime/core/mlas/lib/sqnbitgemm_bitnet_kernel_avx2.cpp
  2. BitNet implementation can be added later.
  3. Tests for mlas function is at onnxruntime\test\mlas\unittest\test_sqnbitgemm.cpp by uncommenting SQNBitGemmShortExecuteTest<2, blklen>::RegisterShortExecuteTests();
  4. matmulnbit kernel implementation is at: onnxruntime\contrib_ops\cpu\quantization\matmul_nbits.cc
  5. tests for matmulnbit kernel are at onnxruntime\test\contrib_ops\matmul_4bits_test.cc by enabling DISABLED_Float32_Accuracy4_Q2.

@liqunfu liqunfu requested a review from a team as a code owner January 30, 2025 03:12
@jywu-msft
Copy link
Member

Can you add description/context ?

@liqunfu liqunfu changed the title bitnet and t-mac (WIP) bitnet and t-mac Jan 31, 2025
github-advanced-security[bot]
github-advanced-security bot found potential problems Jan 31, 2025
View reviewed changes
onnxruntime/core/mlas/lib/q4_dq.cpp
@@ -402,7 +402,8 @@
struct BlockwiseQuantizer {
// To support other qbits, need to add bit packing code for
// storing to dst and zero points
static_assert(qbits == 4, "Only 4b block quantization is supported!");
static_assert(qbits == 4 || qbits == 2, "Only 4b block quantization is supported!");
//static_assert(qbits != 2 || Columnwise, "Only support Columnwise in qbits == 2 case.");

Check notice

Code scanning / CodeQL

Commented-out code

This comment appears to contain commented-out code.
@liqunfu
some fixes
f6f22e3 
Signed-off-by: Liqun Fu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@liqunfu @jywu-msft