Skip to content

Conversation

mitruska
Copy link
Contributor

@mitruska mitruska commented Sep 23, 2025

Details:

  • Fuse vectorized MatMul experts into MOE for 3GEMMs and 2GEMMs pattern:
class ov::pass::VectorizedExpertsFusion : public ov::pass::GraphRewrite {
public:
    OPENVINO_GRAPH_REWRITE_RTTI("VectorizedExpertsFusion");
    VectorizedExpertsFusion() {
        add_matcher<ov::pass::FuseVectorizedMOE2GEMM>();
        add_matcher<ov::pass::FuseVectorizedMOE3GEMM>();
    }
};
  • Add internal MOE op

Example registration of the fusion transformation for CPU plugin: 41145cf

MOE internal op spec PR:

Tickets:

  • transformation (and fusion details): 173663, op: 171913

@mitruska mitruska self-assigned this Sep 23, 2025
@github-actions github-actions bot added category: Core OpenVINO Core (aka ngraph) category: transformations OpenVINO Runtime library - Transformations category: CPP API OpenVINO CPP API bindings labels Sep 23, 2025
@github-actions github-actions bot added category: CPU OpenVINO CPU plugin and removed category: CPP API OpenVINO CPP API bindings labels Sep 24, 2025
@mitruska mitruska changed the title [Transformations][MOE] Fuse vectorized MatMul experts into MOE [Transformations][MOE] Add MOE internal op and fuse vectorized MatMul experts into MOE Sep 24, 2025
@mitruska mitruska marked this pull request as ready for review September 24, 2025 08:50
@mitruska mitruska requested review from a team as code owners September 24, 2025 08:50
@mitruska mitruska requested review from CuriousPanCake and removed request for a team September 24, 2025 08:50
@maxnick maxnick self-assigned this Sep 24, 2025
OV_OP_SCOPE(internal_MOE_validate_and_infer_types);
// TODO: Add inputs validation

set_output_type(0, get_input_element_type(0), get_input_partial_shape(0));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can also use shape of weights for dimension size deduction if some of them is unknown in input hidden_state

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: Core OpenVINO Core (aka ngraph) category: CPU OpenVINO CPU plugin category: transformations OpenVINO Runtime library - Transformations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants