Skip to content

[bug] Loss masking does not happen for sequence_first=True #206

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sohamparikh opened this issue Mar 26, 2025 · 0 comments · Fixed by #207
Closed

[bug] Loss masking does not happen for sequence_first=True #206

sohamparikh opened this issue Mar 26, 2025 · 0 comments · Fixed by #207
Assignees
Labels
bug Something isn't working

Comments

@sohamparikh
Copy link
Member

🐞 Describe the Bug

Loss mask is currently only applied when sequence_first is False. It needs to be enabled for all cases and spans need to be clipped based on sequence_k and sequence_q. This only affects cases when sampling.use_loss_masking_spans=True

if batch.loss_masking_spans is not None:

🎯 Expected Behavior

Loss mask is applied

📝 Additional Context

Loss masking were introduced in #113, but the implementation did not consider sequence_k/sequence_q

@sohamparikh sohamparikh added the bug Something isn't working label Mar 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants