Skip to content

sycl: fix soft_max_f32 max reduction#24451

Merged
ggerganov merged 1 commit into
ggml-org:masterfrom
someoneinjd:master
Jun 15, 2026
Merged

sycl: fix soft_max_f32 max reduction#24451
ggerganov merged 1 commit into
ggml-org:masterfrom
someoneinjd:master

Conversation

@someoneinjd

Copy link
Copy Markdown
Contributor

Overview

Fix the second-stage max reduction in the SYCL softmax kernel when a workgroup contains more subgroups than WARP_SIZE.

Previously, after each subgroup wrote its partial max to buf_iw[warp_id], the cross-subgroup reduction only loaded:

max_val = buf_iw[lane_id];

This only reduces the first WARP_SIZE partial maxima. For example, on Intel GPUs, WARP_SIZE can be 16 while the softmax workgroup size is 1024, producing nwarps = 64. In that case, partial maxima from buf_iw[16..63] were ignored.
If the true row max is in one of the ignored subgroups, softmax subtracts a too-small max value. For large attention logits this can overflow exp(x - max) to inf, eventually producing NaN.

Requirements

@someoneinjd someoneinjd requested a review from a team as a code owner June 11, 2026 03:22
@github-actions github-actions Bot added ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language labels Jun 11, 2026

@arthw arthw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's good job!

Thank you!

@jlionhan

Copy link
Copy Markdown

This PR also fixes a bug in VAE processing of stable-diffusion.cpp that caused fully white image to be generated.

Thank you for your contribution!

@someoneinjd

Copy link
Copy Markdown
Contributor Author

This PR also fixes a bug in VAE processing of stable-diffusion.cpp that caused fully white image to be generated.

Thank you for your contribution!

Haha, I actually discovered this problem when running stable-diffusion.cpp on my Lunar Lake laptop as well. Glad this could help.

Once this PR gets merged into ggml, I will submit an issue to stable-diffusion.cpp requesting them to update ggml.

@arthw arthw added the merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. label Jun 11, 2026
@ggerganov ggerganov merged commit d8a3f52 into ggml-org:master Jun 15, 2026
41 of 54 checks passed
am17an pushed a commit to am17an/llama.cpp that referenced this pull request Jun 20, 2026
papamoose pushed a commit to papamoose/llama.cpp that referenced this pull request Jun 27, 2026
adrianhoehne pushed a commit to adrianhoehne/llama.cpp that referenced this pull request Jul 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants