[distributed] Context parallelism with expert parallelism - #50514

[distributed] Context parallelism with expert parallelism#50514

3outeille wants to merge 2 commits into

cp-6-tp-x-cphuggingface/transformers:cp-6-tp-x-cpfrom

cp-9-ep-x-cphuggingface/transformers:cp-9-ep-x-cpCopy head branch name to clipboard