tests : add -INF blocks to the KQ mask in the FA tests

[ggerganov]

ref #16372 (comment)

The mask that we use in the FA tests is always filled with non-INF values, while in practice it often contains -INF blocks due to causal masking or due to padding. Extend the tests to add -INF values so we can exercise any logic that relies on detecting such blocks in the mask.

[Green-Sky]

I forgot to mention this, but it is possible that cuda fails this test.

[ggerganov]

@Green-Sky Do you mean you tried it locally and it failed?

[Green-Sky]

I just tried it locally and it works. I will have to go and look at the sd.cpp+chroma+fa bug again, so I actually know what fails there.

[Green-Sky]

Ok, did some more debugging. I am not sure if the issue really is with ggml, but flash attention just seems to die when we use the mask in the DiT for chroma.

The mask is:

• 0 in [0;19] (in my example)
• -inf in [20;511]
• and then padded with 0 again for [512;2815]

attention_ext L_q:2816 L_k:2816 n_head:24 C:3072 d_head:128 N:1

[ggerganov]

Does it show any error? What do you mean by "die"?

[Green-Sky]

Does it show any error? What do you mean by "die"?

Ah, sorry for that.
By "die", I mean silently fail. Pulling data from a backend is kind of a pain, but the resulting image is always black, so I assume it it pulled to (+/-) INF somewhere, or maybe zero.