Skip to content

vulkan: handle large sizes for get_rows#15686

Merged
0cc4m merged 1 commit into
ggml-org:masterfrom
jeffbolznv:large_get_rows
Aug 31, 2025
Merged

vulkan: handle large sizes for get_rows#15686
0cc4m merged 1 commit into
ggml-org:masterfrom
jeffbolznv:large_get_rows

Conversation

@jeffbolznv

@jeffbolznv jeffbolznv commented Aug 30, 2025

Copy link
Copy Markdown
Contributor

See #15669.

I'll push tests in a separate PR because I suspect they'll break other backends.

@jeffbolznv jeffbolznv requested a review from 0cc4m as a code owner August 30, 2025 19:15
@github-actions github-actions Bot added Vulkan Issues specific to the Vulkan backend ggml changes relating to the ggml tensor library for machine learning labels Aug 30, 2025
This was referenced Aug 30, 2025
Merged
Merged
@0cc4m

0cc4m commented Aug 31, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

I see no issues. Looks good.

gpu_info backends model_type model_size fa test avg_ts(master) avg_ts(pr) %
AMD Radeon (TM) Pro VII (RADV VEGA20) Vulkan gpt-oss 20B Q8_0 11.27 GiB 0 pp512 573.07 572.15 -0.2%
AMD Radeon (TM) Pro VII (RADV VEGA20) Vulkan gpt-oss 20B Q8_0 11.27 GiB 0 tg128 108.78 109.03 +0.2%
AMD Radeon (TM) Pro VII (RADV VEGA20) Vulkan llama 13B Q4_0 12.56 GiB 0 pp512 260.80 260.26 -0.2%
AMD Radeon (TM) Pro VII (RADV VEGA20) Vulkan llama 13B Q4_0 12.56 GiB 0 tg128 26.62 26.55 -0.2%
AMD Radeon (TM) Pro VII (RADV VEGA20) Vulkan llama 7B Q4_0 3.56 GiB 0 pp512 835.60 831.11 -0.5%
AMD Radeon (TM) Pro VII (RADV VEGA20) Vulkan llama 7B Q4_0 3.56 GiB 0 tg128 80.48 80.30 -0.2%
AMD Radeon (TM) Pro VII (RADV VEGA20) Vulkan llama 8B Q4_K - Small 4.36 GiB 0 pp512 293.51 292.27 -0.4%
AMD Radeon (TM) Pro VII (RADV VEGA20) Vulkan llama 8B Q4_K - Small 4.36 GiB 0 tg128 71.97 71.54 -0.6%
Intel(R) Arc(tm) A770 Graphics (DG2) Vulkan gpt-oss 20B Q8_0 11.27 GiB 0 pp512 184.80 184.76 -0.0%
Intel(R) Arc(tm) A770 Graphics (DG2) Vulkan gpt-oss 20B Q8_0 11.27 GiB 0 tg128 20.03 20.02 -0.1%
Intel(R) Arc(tm) A770 Graphics (DG2) Vulkan llama 13B Q4_0 12.56 GiB 0 pp512 276.56 273.86 -1.0%
Intel(R) Arc(tm) A770 Graphics (DG2) Vulkan llama 13B Q4_0 12.56 GiB 0 tg128 16.47 16.49 +0.1%
Intel(R) Arc(tm) A770 Graphics (DG2) Vulkan llama 7B Q4_0 3.56 GiB 0 pp512 658.03 657.06 -0.1%
Intel(R) Arc(tm) A770 Graphics (DG2) Vulkan llama 7B Q4_0 3.56 GiB 0 tg128 46.83 46.72 -0.2%
Intel(R) Arc(tm) A770 Graphics (DG2) Vulkan llama 8B Q4_K - Small 4.36 GiB 0 pp512 100.39 100.39 -0.0%
Intel(R) Arc(tm) A770 Graphics (DG2) Vulkan llama 8B Q4_K - Small 4.36 GiB 0 tg128 30.55 30.52 -0.1%
NVIDIA GeForce RTX 3090 Vulkan gpt-oss 20B Q8_0 11.27 GiB 0 pp512 3832.77 3816.03 -0.4%
NVIDIA GeForce RTX 3090 Vulkan gpt-oss 20B Q8_0 11.27 GiB 0 tg128 147.77 146.95 -0.6%
NVIDIA GeForce RTX 3090 Vulkan llama 13B Q4_0 12.56 GiB 0 pp512 1722.12 1718.14 -0.2%
NVIDIA GeForce RTX 3090 Vulkan llama 13B Q4_0 12.56 GiB 0 tg128 52.11 51.55 -1.1%
NVIDIA GeForce RTX 3090 Vulkan llama 7B Q4_0 3.56 GiB 0 pp512 4406.60 4371.66 -0.8%
NVIDIA GeForce RTX 3090 Vulkan llama 7B Q4_0 3.56 GiB 0 tg128 143.50 143.31 -0.1%
NVIDIA GeForce RTX 3090 Vulkan llama 8B Q4_K - Small 4.36 GiB 0 pp512 4492.99 4488.97 -0.1%
NVIDIA GeForce RTX 3090 Vulkan llama 8B Q4_K - Small 4.36 GiB 0 tg128 118.45 117.96 -0.4%

@0cc4m 0cc4m merged commit bbbf5ec into ggml-org:master Aug 31, 2025
46 of 48 checks passed
walidbr pushed a commit to walidbr/llama.cpp that referenced this pull request Sep 7, 2025
@jeffbolznv @walidbr
vulkan: handle large sizes for get_rows (ggml-org#15686)
e16026b
blime4 referenced this pull request in blime4/llama.cpp Feb 5, 2026
@jeffbolznv
vulkan: handle large sizes for get_rows (#15686)
5930329
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
@jeffbolznv
vulkan: handle large sizes for get_rows (ggml-org#15686)
b31892e
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
@jeffbolznv
vulkan: handle large sizes for get_rows (ggml-org#15686)
7834b48
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
@jeffbolznv
vulkan: handle large sizes for get_rows (ggml-org#15686)
e7c4bb7
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
@jeffbolznv
vulkan: handle large sizes for get_rows (ggml-org#15686)
a32d08d
phibya pushed a commit to ziee-ai/llama.cpp that referenced this pull request May 29, 2026
@jeffbolznv
vulkan: handle large sizes for get_rows (ggml-org#15686)
87b89d1
fewtarius pushed a commit to fewtarius/CachyLLama that referenced this pull request May 30, 2026
@jeffbolznv
vulkan: handle large sizes for get_rows (ggml-org#15686)
49d5db4
fukuro-kun pushed a commit to fukuro-kun/fukuro-llama-cpp-turboquant that referenced this pull request Jul 5, 2026
@jeffbolznv
vulkan: handle large sizes for get_rows (ggml-org#15686)
085d516
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@0cc4m 0cc4m 0cc4m approved these changes

Assignees

No one assigned

Labels

ggml changes relating to the ggml tensor library for machine learning Vulkan Issues specific to the Vulkan backend

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jeffbolznv @0cc4m