Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bump Megatron LM example from 0.4 to 0.8.0 (or latest as option) #370

Closed
perifaws opened this issue Jul 3, 2024 · 1 comment · Fixed by #437
Closed

Bump Megatron LM example from 0.4 to 0.8.0 (or latest as option) #370

perifaws opened this issue Jul 3, 2024 · 1 comment · Fixed by #437
Assignees
@amanshanbhag
Labels
enhancement New feature or request

Comments

@perifaws
Copy link
Contributor

perifaws commented Jul 3, 2024 •
edited
Loading

The MegatronLM sample is set to 0.4 and needs an update to a later version starting at 0.7. The version should be configurable with a default in the docker file (top).

This is the part to update but there will be dependencies changes, for example on transformers and NCCL + OFI.

RUN cd /workspace && git clone --depth 1 --branch core_v0.4.0 https://github.com/NVIDIA/Megatron-LM.git \
	&& cd Megatron-LM \
	&& python3 -m pip install nltk  \
	&& python -m pip install .
@perifaws perifaws added the enhancement New feature or request label Jul 3, 2024
@mhuguesaws
Copy link
Contributor

This requires actions due to 1y+ packages.

@mhuguesaws mhuguesaws mentioned this issue Sep 20, 2024
Merged
@mhuguesaws mhuguesaws changed the title Bump Megatron LM example from 0.4 to 0.7 (or latest as option) Bump Megatron LM example from 0.4 to 0.8.0 (or latest as option) Sep 20, 2024
mhuguesaws added a commit that referenced this issue Sep 20, 2024
@mhuguesaws
Improvements/#370 bump megatron version (#437)
517cb39 
* Change nvidia pytorch to 24.08

* Change package version and clean up

* Change ocntainer to define megatron core and transformers versions

* Add OMPI and NCCL environment variables

* Fix pkg_resources for setuptools 70+
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@amanshanbhag amanshanbhag

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Improvements/#370 bump megatron version aws-samples/awsome-distributed-training
3 participants
@amanshanbhag @perifaws @mhuguesaws