Skip to content

[MOE] Fix issues with batch, use postponed_const when fusing expert weights #32546

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open

[MOE] Fix issues with batch, use postponed_const when fusing expert weights #32546

wants to merge 9 commits into from
+155 −116

Conversation

@mmikolajcz
Copy link
Contributor

Details:

  • Use postponed constant instead make try fold to limit memory usage when folding large concatenation
  • Fix issue where inference would fail in batch sizes different than 1 due to incorrect reshape target

Tickets:

  • ticket-id

@mmikolajcz mmikolajcz added this to the 2025.4 milestone Oct 24, 2025
@github-actions github-actions bot added category: transformations OpenVINO Runtime library - Transformations category: TF FE OpenVINO TensorFlow FrontEnd category: PyTorch FE OpenVINO PyTorch Frontend category: JAX FE OpenVINO JAX FrontEnd labels Oct 24, 2025
@github-actions github-actions bot added category: CI OpenVINO public CI github_actions Pull requests that update GitHub Actions code labels Oct 27, 2025
@mmikolajcz mmikolajcz marked this pull request as ready for review October 28, 2025 08:13
@mmikolajcz mmikolajcz requested review from a team as code owners October 28, 2025 08:13
@mmikolajcz mmikolajcz requested review from CuriousPanCake and removed request for a team October 28, 2025 08:13
mryzhov
mryzhov reviewed Oct 28, 2025
View reviewed changes
.github/workflows/job_pytorch_models_tests.yml
if: ${{ inputs.model_scope == 'precommit' }}
run: |
export PYTHONPATH=${MODEL_HUB_TESTS_INSTALL_DIR}:$PYTHONPATH
python3 -m pip install optimum-intel==1.25.2 transformers==4.53.3 --upgrade
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a bad practice to introduce test dependencies in the pipeline, why can't we use the common test requirements?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upgrade to transformers and optimum-intel is required to use versions that support qwen3_moe models.
Ideally it would be best to upgrade tests/requirements_pytorch to align versions, however this might be bit risky without thorough testing since this requirements are used in large amount of model tests, including nightly large models.
I've initially tried running other precommit suites test_pa_transformation test_transformations and it seem they were successful with upgraded versions

@moslex moslex added the priority: high High piority label Oct 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mryzhov mryzhov mryzhov approved these changes

@CuriousPanCake CuriousPanCake Awaiting requested review from CuriousPanCake CuriousPanCake is a code owner automatically assigned from openvinotoolkit/openvino-ie-transformations-maintainers

Assignees

No one assigned

Labels

category: CI OpenVINO public CI category: JAX FE OpenVINO JAX FrontEnd category: PyTorch FE OpenVINO PyTorch Frontend category: TF FE OpenVINO TensorFlow FrontEnd category: transformations OpenVINO Runtime library - Transformations Code Freeze github_actions Pull requests that update GitHub Actions code priority: high High piority

Projects

None yet

Milestone

2025.4

Development

Successfully merging this pull request may close these issues.

3 participants

@mmikolajcz @mryzhov @moslex