[RELEASE] Fix compression of first & last layers of bitnet to int8 (#3738) #3739
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…lkit#3738) ### Changes Added u2 type support in NNCF graph ### Reason for changes int8 compression of first/last layer of bitnet fails with error: ``` raise NotImplementedError(msg) NotImplementedError: NNCF is not yet supported OpenVINO data type: u2. ``` ### Related tickets CVS-176501 ### Tests manual, since need optimum from main branch and openvino==2025.4 and OV doesn't expose `ov.Type.u2` in Python API (requested for 2026.0 release) ```shell $ pip install git+https://github.com/huggingface/optimum-intel.git $ pip install --pre openvino==2025.4.0rc2 openvino-tokenizers==2025.4.0.0rc2 openvino-genai==2025.4.0.0rc2 --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly $ optimum-cli export openvino -m optimum-intel-internal-testing/tiny-random-bitnet --weight-format int8 out_dir --task=text-generation-with-past $ optimum-cli export openvino -m microsoft/bitnet-b1.58-2B-4T --weight-format int8 out_dir --task=text-generation-with-past ``` <img width="1024" height="143" alt="image" src="https://github.com/user-attachments/assets/ffd2b816-88d3-4a35-bc61-5048f7cc4f9b" />
nikita-savelyevv
approved these changes
Nov 17, 2025
MaximProshin
changed the title
MaximProshin
approved these changes
Nov 19, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#3738 to the release branch
The fix is pretty-simple, but works with OpenVINO 2025.4
Once optimum is released, should work out-of-the box
Changes
Added u2 type support in NNCF graph
Reason for changes
int8 compression of first/last layer of bitnet fails with error:
Related tickets
CVS-176501
Tests
conformance weight compression on after #3738 merge: https://github.com/openvinotoolkit/nncf/actions/runs/19437926992
conformance quantization on after #3738 merge: manual/job/post_training_quantization/755/
manual, since need optimum from main branch and openvino==2025.4 and OV doesn't expose
ov.Type.u2in Python API (requested for 2026.0 release)