From NVIDIA Megatron-LM for visibility #18
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: lit <[email protected]> Co-authored-by: Yuzhong Wang <[email protected]>
feat(MoE): support CP and recompute for MTP See merge request ADLR/megatron-lm!3330
Disallow expandable segments for cudagraphs entirely See merge request ADLR/megatron-lm!3806
MXFP8 DP AG overlap enablement See merge request ADLR/megatron-lm!3710
Signed-off-by: oliver könig <[email protected]>
Signed-off-by: oliver könig <[email protected]>
Co-authored-by: Santosh Bhavani <[email protected]>
Update README See merge request ADLR/megatron-lm!3406
Move FullCudaGraphWrapper implementation to Megatron Core. See merge request ADLR/megatron-lm!3808
Fixes and updates for external cudagraph See merge request ADLR/megatron-lm!3631
build: Bump TE See merge request ADLR/megatron-lm!3799
Debug distributed checkpoint for Transformer Engine fused MLP See merge request ADLR/megatron-lm!3606
Add argument to control collnet enablement See merge request ADLR/megatron-lm!3812
Dynamic Backend Inference MLA See merge request ADLR/megatron-lm!3569
Co-authored-by: Mcore Bot <[email protected]> Co-authored-by: Brandon Norick <[email protected]>
Adding support for multiple validation sets See merge request ADLR/megatron-lm!3422
…m, and sequence parallelism for dynamic engine
Fix log prob calculation, pipeline parallelism, and sequence parallelism for dynamic engine See merge request ADLR/megatron-lm!3718
…calling the right fp8_context
Fix bug in param_norm computation where some ranks might call collective and some might not See merge request ADLR/megatron-lm!3918
Fix BERT + virtual pipeline parallelism See merge request ADLR/megatron-lm!3993
…phs. Co-authored-by: Mcore Bot <[email protected]> Co-authored-by: Siddharth Singh <[email protected]>
Dynamic inference functional tests | Cuda graphs. See merge request ADLR/megatron-lm!3620
… training graph creation until create_cudagraphs
…main' Create inference graphs immediately but defer training graph creation until create_cudagraphs See merge request ADLR/megatron-lm!3965
Set mimo_vlm and gpt_dynamic_inference tests to be flaky See merge request ADLR/megatron-lm!3995
Signed-off-by: oliver könig <[email protected]>
Signed-off-by: oliver könig <[email protected]>
Signed-off-by: oliver könig <[email protected]>
Co-authored-by: Mcore Bot <[email protected]>
chore: Upgrade dependencies (2025-09-15) See merge request ADLR/megatron-lm!3998
Co-authored-by: vignesh1507 <[email protected]>
Co-authored-by: Mcore Bot <[email protected]>
Co-authored-by: John Kamalu <[email protected]>
…ate to safe globals Co-authored-by: Mcore Bot <[email protected]>
Co-authored-by: Jon Barker <[email protected]> Co-authored-by: Robert Kirby <[email protected]> Co-authored-by: Vitaly Kurin <[email protected]> Co-authored-by: Helen Ngo <[email protected]> Co-authored-by: Mcore Bot <[email protected]> Co-authored-by: Keshav Santhanam <[email protected]> Co-authored-by: Robert Kirby <[email protected]>
…fline implementation Co-authored-by: Chenhan Yu <[email protected]> Co-authored-by: Oliver Koenig <[email protected]> Co-authored-by: Ye Yu <[email protected]> Co-authored-by: Ye Yu <[email protected]> Co-authored-by: Ye Yu <[email protected]>
… but only on the last layer
… Gradient Clipping Co-authored-by: Wil Kong <[email protected]>
Co-authored-by: Selvaraj Anandaraj <[email protected]>
… fully_shard_model and fully_shard_optimizer. Co-authored-by: Mcore Bot <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.