CUDA: update build CTK version to 12.8

[thevishalagarwal]

update CUDA Toolkit version from 12.4 to 12.8 to support compilation of real arch sm120 Blackwell GPUs.

• updated ggml-cuda/CMakeLists.txt to add compilation of sm120 arch for Blackwell GPUs
• updated CUDAToolkit version from 12.4 to 12.8 for windows github CI build

[thevishalagarwal]

@JohannesGaessler @slaren @ggerganov ping for review

[slaren]

I am not sure that we need to add real arch 120 to the build. The criteria for selecting the real archs to include is based on what we expect to be the most commonly used GPUs to improve the load time in these cases, but at this point there are likely very few people with RTX 50 series GPUs, below 1% according to the steam hw survey.

[thad0ctor]

I am not sure that we need to add real arch 120 to the build. The criteria for selecting the real archs to include is based on what we expect to be the most commonly used GPUs to improve the load time in these cases, but at this point there are likely very few people with RTX 50 series GPUs, below 1% according to the steam hw survey.

why not? The percantage of people using blackwell for ai, particularly 5090s and RTX 6000 are probably disproportionate to the overall steam hw survey

[thad0ctor]

Is there any reason this is still hanging out there?

[Thireus]

Boycott? 🧐

[thad0ctor]

I am not sure that we need to add real arch 120 to the build. The criteria for selecting the real archs to include is based on what we expect to be the most commonly used GPUs to improve the load time in these cases, but at this point there are likely very few people with RTX 50 series GPUs, below 1% according to the steam hw survey.

@JohannesGaessler @slaren @ggerganov

This commit resolves t/s speed regressions introduced by commit 225e7a1 - 225e7a1 . These speed regressions are described here: #14881 and here #14795 (comment)

After investigating this issue, the 4D FlashAttention changes in commit 225e7a1 have compile-time architecture dependencies that prevent them from working properly on RTX 5090/5xxxx systems. Unlike previous versions where
manual -DCMAKE_CUDA_ARCHITECTURES="120" overrides were sufficient, the new 4D implementation requires compute capability 12.0 to be explicitly included in the default CMakeLists.txt architecture configuration or it defaults to a comiling with lesser performance. The 4D code contains compile-time checks and template instantiations that only generate the necessary kernel variants when 120-real is present in the default build targets - manual cmake overrides no longer work because the required code paths simply aren't compiled.

Is there a way we can fast track the merging of this PR? Without this PR I get 40 t/s on Qwen3 235B q3, after merging this PR on my own I get >57 t/s? Thank you!

[thad0ctor]

Boycott? 🧐

Hopefully no longer, I just looked at the current steam survey and RTX 5XXX cards account for over 3.5% as of June:https://store.steampowered.com/hwsurvey/videocard/

This PR also fixes speed issues for RTX 5xxx cards introduced mid-month, hopefully my last post helps move it along!