Repository navigation
Conversation
|
[For maintainers] Suggested jobs to run (before merge) run-slow: bamba, bark, clipseg, efficientloftr, falcon, higgs_audio_v2_tokenizer, idefics2, mimi, nemotron3_5_transcribe, squeezebert |
|
run-slow: bamba, bark, clipseg, efficientloftr, falcon, higgs_audio_v2_tokenizer, idefics2, mimi, nemotron3_5_transcribe, squeezebert |
| # Beware that with flash_attn<2.1, using q_seqlen != k_seqlen (except for the case q_seqlen == 1) produces a wrong mask (top-left). | ||
| self._flash_attn_uses_top_left_mask = flash_attn_supports_top_left_mask() | ||
|
|
||
| def _split_heads(self, tensor, num_heads, attn_head_size): |
There was a problem hiding this comment.
this was an actual error and the shape was destroying cache updates
|
|
||
| def _test_attention_implementation(self, attn_implementation): | ||
| """Audio encoder can produce fp32 (attention) outputs => avoid by autocasting""" | ||
| with torch.autocast(device_type=torch_device, dtype=torch.float16): |
There was a problem hiding this comment.
I opted for autocast for now but maybe in the end it would be best to fix it in modeling directly cc @eustlb
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
CI (Nvidia + AMD)Nvidia · ✅ No new failures 🎉👏 · ⚙️ Workflow · 📊 Dashboard Failures (5 pre-existing)Pre-existing failures (still failing) — 5 total: Same error (5): - bamba: - falcon: - mimi: AMD · ✅ No new failures 🎉👏 · ⚙️ Workflow · 📊 Dashboard Failures (9 pre-existing)Pre-existing failures (still failing) — 9 total: Same error (9): - bamba: - efficientloftr: - falcon: - mimi: - nemotron3_5_transcribe: |
| def test_flash_attn_2_inference_equivalence(self): | ||
| for model_class in self.all_model_classes: | ||
| if not model_class._supports_flash_attn: | ||
| self.skipTest(f"{model_class.__name__} does not support Flash Attention 2") |
There was a problem hiding this comment.
you are looping all_model_classes and doign a skip inside, which exits the whole test. If the intent is to skip only unsupported classes, filtering the classes or using subTest() would work better
As per title