Commit 8f4c127
[ROCm][Bugfix] Use BF16 MLA prefill for short prompts
Avoid FP8 rounding perturbations on short greedy-decoding requests while preserving the optimized AITER path for long-context prefill.
Co-authored-by: Cursor <cursoragent@cursor.com>1 parent beca88e commit 8f4c127
1 file changed
Lines changed: 3 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1035 | 1035 | | |
1036 | 1036 | | |
1037 | 1037 | | |
1038 | | - | |
| 1038 | + | |
| 1039 | + | |
| 1040 | + | |
1039 | 1041 | | |
1040 | 1042 | | |
1041 | 1043 | | |
| |||
0 commit comments