Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements/#370 bump megatron version #437

Merged
merged 5 commits into from
Sep 20, 2024

Conversation

mhuguesaws
Copy link
Contributor

@mhuguesaws mhuguesaws commented Sep 20, 2024

Issue #, if available:
Close #370

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@shimomut
Copy link
Collaborator

Got this error when I ran "sbatch 1.data-preprocessing.sbatch".

0: Traceback (most recent call last):
0:   File "/workspace/Megatron-LM/tools/preprocess_data.py", line 23, in <module>
0:     from megatron.training.tokenizer import build_tokenizer
0:   File "/workspace/Megatron-LM/megatron/training/__init__.py", line 5, in <module>
0:     from .global_vars import get_args
0:   File "/workspace/Megatron-LM/megatron/training/global_vars.py", line 9, in <module>
0:     from megatron.core import Timers, init_num_microbatches_calculator
0:   File "/usr/local/lib/python3.10/dist-packages/megatron/core/__init__.py", line 1, in <module>
0:     import megatron.core.tensor_parallel
0:   File "/usr/local/lib/python3.10/dist-packages/megatron/core/tensor_parallel/__init__.py", line 3, in <module>
0:     from .layers import (
0:   File "/usr/local/lib/python3.10/dist-packages/megatron/core/tensor_parallel/layers.py", line 39, in <module>
0:     from .random import get_cuda_rng_tracker, get_expert_parallel_rng_tracker_name
0:   File "/usr/local/lib/python3.10/dist-packages/megatron/core/tensor_parallel/random.py", line 10, in <module>
0:     from pkg_resources import packaging
0: ImportError: cannot import name 'packaging' from 'pkg_resources' (/usr/local/lib/python3.10/dist-packages/pkg_resources/__init__.py)

@mhuguesaws mhuguesaws force-pushed the improvements/#370_bump_megatron_version branch 2 times, most recently from 112c6dd to 8778671 Compare September 20, 2024 18:46
@shimomut
Copy link
Collaborator

Checking if "pip3 install setuptools==75.1.0 --upgrade" can be a simpler solution than patching megatron.

@mhuguesaws mhuguesaws force-pushed the improvements/#370_bump_megatron_version branch from 8778671 to 3610896 Compare September 20, 2024 18:50
Copy link
Collaborator

@shimomut shimomut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@mhuguesaws mhuguesaws marked this pull request as ready for review September 20, 2024 19:33
@mhuguesaws mhuguesaws merged commit 517cb39 into main Sep 20, 2024
@mhuguesaws mhuguesaws deleted the improvements/#370_bump_megatron_version branch September 20, 2024 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bump Megatron LM example from 0.4 to 0.8.0 (or latest as option)
2 participants