scr: initial integration for Microsoft Megatron-Deepspeed#5

Open

adammoody wants to merge 25 commits intoms-mainfrom

Commits on Nov 3, 2022

Modifying loss checking to support bf16. (#92)
jomayeri
authored

Commits on Nov 7, 2022

Updated to Curated acpt env and removed deepspeed install from github (#88)
savitamittal1
authored

Commits on Nov 18, 2022

Fix the bug of FusedLayerNorm on ROCm (#96)
hubertlu-tw
authored

Commits on Dec 7, 2022

Fix generate_text.sh Megatron text-generation example working w/ DS inference (#99)
lekurile
authored

Commits on Dec 12, 2022

DeepSpeed Data Efficiency Library pretraining examples (#100)
conglongli
authored

Commits on Dec 14, 2022

fix floating point in script (#101)
conglongli
authored

Commits on Dec 19, 2022

added unit test for megatron (#102)
mrwyattii
authored

Commits on Dec 20, 2022

fix deprecated numpy types (#103)
mrwyattii
authored

Commits on Dec 23, 2022

fix script typo
conglongli
authored

Commits on Jan 30, 2023

Use accelerator for supporting other device (#104)
inkcherry
authored

Commits on Feb 21, 2023

data efficiency example update (#113)
conglongli
authored

Commits on Mar 15, 2023

fix torch six import error (#119)
ajindal1
authored

Commits on Apr 29, 2023

fix(uitls.py): add () after device_name (#130)
gfzum
authored

Commits on May 1, 2023

Remove duplicate content in README (#131)
sbmaruf
authored

Commits on May 5, 2023

fix a bug when run on bf16+pp (#134)
ys950902
authored

Commits on May 6, 2023

Add a space to fix tab error for loss_scale (#136)
ys950902
authored

Commits on May 10, 2023

fix(training.py): logical bug in eval_iters_calculation (#138)
2catycm
authored

Commits on Jun 8, 2023