Skip to content

Commit e5259e4

Browse files
authored
[RELEASE] Fix compression of first & last layers of bitnet to int8 (#3738) (#3739)
#3738 to the release branch The fix is pretty-simple, but works with OpenVINO 2025.4 Once optimum is released, should work out-of-the box ### Changes Added u2 type support in NNCF graph ### Reason for changes int8 compression of first/last layer of bitnet fails with error: ``` raise NotImplementedError(msg) NotImplementedError: NNCF is not yet supported OpenVINO data type: u2. ``` ### Related tickets CVS-176501 ### Tests conformance weight compression on after #3738 merge: https://github.com/openvinotoolkit/nncf/actions/runs/19437926992 conformance quantization on after #3738 merge: manual/job/post_training_quantization/755/ manual, since need optimum from main branch and openvino==2025.4 and OV doesn't expose `ov.Type.u2` in Python API (requested for 2026.0 release) ```shell $ pip install git+https://github.com/huggingface/optimum-intel.git $ pip install --pre openvino==2025.4.0rc2 openvino-tokenizers==2025.4.0.0rc2 openvino-genai==2025.4.0.0rc2 --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly $ optimum-cli export openvino -m optimum-intel-internal-testing/tiny-random-bitnet --weight-format int8 out_dir --task=text-generation-with-past $ optimum-cli export openvino -m microsoft/bitnet-b1.58-2B-4T --weight-format int8 out_dir --task=text-generation-with-past ``` <img width="1024" height="143" alt="image" src="https://github.com/user-attachments/assets/ffd2b816-88d3-4a35-bc61-5048f7cc4f9b" />
1 parent 473ef32 commit e5259e4

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

src/nncf/openvino/graph/nncf_graph_builder.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@ def convert_to_nncf_dtype(ov_type: ov.Type) -> Dtype:
5858
"i32": "int",
5959
"i64": "int",
6060
"u1": "int",
61+
"u2": "int",
6162
"u4": "int",
6263
"u8": "int",
6364
"u16": "int",

0 commit comments

Comments
 (0)