Enable T2U Finetuning for UnitYNART2UModel in SeamlessM4T_v2_large and Refactor Context Management#563
Enable T2U Finetuning for UnitYNART2UModel in SeamlessM4T_v2_large and Refactor Context Management#563Nabarup1 wants to merge 1 commit intofacebookresearch:mainfrom
Conversation
…); use nullcontext for dummy_context in trainer.py This update enables finetuning of the T2U component for SeamlessM4T_v2_large models that use the UnitYNART2UModel, which was previously unsupported in the official trainer. The UnitYFinetuneWrapper now detects and handles UnitYNART2UModel in addition to UnitYT2UModel, allowing T2U finetuning for v2 models. Additionally, dummy_context has been replaced with nullcontext() to prevent attribute errors during context management. These changes make it possible to run T2U finetuning workflows for SeamlessM4T_v2_large without runtime errors or missing model support.
|
@samueljamesbell @Celebio @mhlakhani |
|
Thank you, that saved me a lot of time, but I made a change in line 334 to make it work. |
|
Appreciate you testing this out, @Omaralsaabi — thanks for pointing out the |
|
Hey @samueljamesbell @Celebio @mhlakhani — just circling back on this. Checks are all green, and with @Omaralsaabi's feedback addressed it should be good to go. Would be great to get a review/approval when you get a chance 🙏 |
This pull request introduces support for finetuning the T2U (Text-to-Unit) component in SeamlessM4T_v2_large models that utilize the UnitYNART2UModel, which was previously not supported in the official finetuning trainer.py. The UnitYFinetuneWrapper has been extended to detect and handle both UnitYT2UModel (v1) and UnitYNART2UModel (v2), enabling comprehensive T2U finetuning for v2 models. Additionally, the dummy_context has been replaced with nullcontext() to ensure robust context management and prevent attribute errors during training. These changes ensure that users can now reliably finetune the T2U module in SeamlessM4T_v2_large without encountering runtime errors or missing model support.
What has been done:
UnitYNART2UModelin the T2U finetuning workflow.UnitYNART2UModel.dummy_contextwithnullcontext()throughout the trainer to prevent context-related attribute errors.