Misc. bug: cpy.cu:229: GGML_ASSERT(grid_z < USHRT_MAX) failed

[askmyteapot]

Name and Version

b9489
This is the latest commit pulled into KoboldCPP (concedo_experimental)

Operating systems

Windows

Which llama.cpp modules do you know to be affected?

No response

Command line

n/a

Problem description & steps to reproduce

Discovered a bug in sd.cpp/koboldcpp when running a workflow to generate video with LTX 2.3

When the resolution and number of frames goes over a certain threshold, it triggers an assert:
ggml\src\ggml-cuda\cpy.cu:229: GGML_ASSERT(grid_z < USHRT_MAX) failed

It appears to be a bug in GGML.

With the assistance of Deepseek v4 Flash, the issue seems to be as below: (sorry for the LLM output, but i'm not very skilled when it comes to this level of coding)

What the assertion means
Line 226 computes grid_z = (ne/(ne01n*ne00n) + 7) / 8. For a contiguous tensor with ne[0]=1 (a single-element first dimension), can_be_transposed evaluates to true, and the transposed copy path uses grid_z as the 3rd CUDA grid dimension. CUDA limits grid_z to 65535 (USHRT_MAX). This assertion fires when ne[2] >= 524,281.

Root Cause: LTX video attention flattens tokens×heads together
In ltxv.hpp:345, apply_hidden_rope flattens tokens and attention heads:

auto x_flat = ggml_reshape_4d(ctx, x4, dim_head, 1, x->ne[1] * heads, x->ne[2]);
For your video (latent 39×26×17 = 17,238 tokens) with 32 attention heads:

x_flat->ne[2] = 17238 × 32 = 551,616
This goes into Rope::apply_rope (rope.hpp:768-771) which does permute+reshape+cont operations. After permutations, the tensor becomes [1, 2, 64, 551616] — making ne[0]=1 with ne[2] >= 524,281, which triggers the assertion in ggml_cont → ggml_cuda_cpy.

Why this is a bug
The ggml_cpy_scalar_cuda transposed path uses grid_z directly as a CUDA grid dimension, but CUDA caps grid y/z at 65535. The assertion correctly catches that the launch would fail, but the underlying issue is that the tensor layout shouldn't go through this code path for such large dimensions.

Fix Options
Option A — Patch cpy.cu: After the assert, fall back to the non-transposed (1D grid) path when grid_z >= USHRT_MAX:

if (grid_z >= USHRT_MAX) {
    // fall through to non-transposed path below
} else {
    dim3 dimGrid(grid_x, grid_y, grid_z);
    ...
    return;
}
// non-transposed path

I tested a patch which worked and will provide in the second message.

Occam has suggested this may be a bug in the cuda path, rather than in SD.CPP as the other workaround was to make the graph worse within sd.cpp.

First Bad Commit

unknown

Relevant log output

Logs

C:\koboldcpp>koboldcpp.py
***
Welcome to KoboldCpp - Version 1.115
For command line arguments, please run --help in the terminal.
Note: The GUI mode is not accessible to screen readers.
***
Auto Selected CUDA Backend (flag=0)

Loading Chat Completions Adapter: C:\koboldcpp\kcpp_adapters\AutoGuess.json
Chat Completions Adapter Loaded
gemma-3-12b-it-Q4_K_S.gguf already exists, using existing file.
Auto Set Threads: 15
System: Windows 10.0.26200 AMD64 AMD64 Family 25 Model 33 Stepping 2, AuthenticAMD
Detected Available GPU Memory: 24576 MB
Detected Available RAM: 55005 MB
Initializing dynamic library: koboldcpp_cublas.dll
==========
Namespace(model=[], model_param=None, port=5001, port_param=5001, host='', launch=False, config=None, threads=15, usecuda=['normal', '1'], usevulkan=None, usecpu=False, contextsize=8192, gpulayers=-1, tensor_split=None, autofit=False, version=False, analyze='', maingpu=-1, batchsize=512, parallelrequests=1, blasthreads=None, splitmode='layer', nommq=False, lora=None, loramult=1.0, noshift=False, nofastforward=False, noswa=False, swapadding=0, smartcache=0, ropeconfig=[0.0, 10000.0], overridenativecontext=0, usemmap=False, usemlock=False, noavx2=False, failsafe=False, debugmode=1, onready='', benchmark=None, prompt='', cli=False, genlimit=0, multiuser=10, multiplayer=False, websearch=False, remotetunnel=False, highpriority=False, foreground=False, preloadstory=None, savedatafile=None, quiet=False, ssl=None, nocertify=False, mmproj=None, mmprojcpu=False, visionmaxres=1024, visionmintokens=-1, visionmaxtokens=-1, draftmodel=None, draftamount=8, draftgpulayers=999, draftgpusplit=None, password=None, ratelimit=0, ignoremissing=False, chatcompletionsadapter='AutoGuess', jinja=False, jinja_tools=False, jinja_kwargs='', jinjatemplate='', jinjathink='default', noflashattention=False, lowvram=False, quantkv='f16', smartcontext=False, unpack='', exportconfig='', exporttemplate='', nomodel=False, moeexperts=-1, moecpu=0, defaultgenamt=1024, nobostoken=False, enableguidance=False, maxrequestsize=32, overridekv=None, overridetensors=None, showgui=False, skiplauncher=False, singleinstance=False, nopipelineparallel=False, gendefaults='', gendefaultsoverwrite=False, mcpfile=None, device='', downloaddir='', autofitpadding=1024, hordemodelname='', hordeworkername='', hordekey='', hordemaxctx=0, hordegenlen=0, sdmodel='C:/LTX2_3/ltx-2.3-22b-dev-UD-Q5_K_S.gguf', sdthreads=15, sdclamped=0, sdclampedsoft=0, sdt5xxl='', sdclip1='gemma-3-12b-it-Q4_K_S.gguf', sdclip2='C:/LTX2_3/ltx-2.3-22b-dev_embeddings_connectors.safetensors', sdphotomaker='', sdupscaler='', sdflashattention=True, sdoffloadcpu=True, sdvaedevice=-1, sdclipdevice=-1, sdconvdirect='off', sdvae='C:/LTX2_3/ltx-2.3-22b-dev_video_vae.safetensors', sdvaeauto=False, sdaudiovae='C:/LTX2_3/ltx-2.3-22b-dev_audio_vae.safetensors', sdquant=0, sdlora=[], sdloramult=[1.0], sdtiledvae=512, sdmaingpu=-1, sdvramlimit=0, whispermodel='', ttsmodel='', ttswavtokenizer='', ttsgpu=False, ttsmaxlen=4096, ttsthreads=15, ttsdir='', musicllm='', musicembeddings='', musicdiffusion='', musicvae='', musiclowvram=False, embeddingsmodel='', embeddingsmaxctx=0, embeddingsgpu=False, rpcmode='disabled', rpcport=5551, rpchost='0.0.0.0', rpcdevice='', rpctargets='', admin=False, adminpassword='', admindir='', adminunloadtimeout=0, routermode=False, reqtimeout=600, autoswapmode=False, baseconfig='', hordeconfig=None, sdconfig=None, noblas=False, nommap=False, pipelineparallel=False, sdnotile=False, sdvaecpu=False, sdclipgpu=False, forceversion=False, sdgendefaults=False, flashattention=False, useswa=False, testmemory=False, proxy_port=None)
==========

ImageGen Init - Load Model: C:\LTX2_3\ltx-2.3-22b-dev-UD-Q5_K_S.gguf
With Custom VAE: C:\LTX2_3\ltx-2.3-22b-dev_video_vae.safetensors
With Audio VAE: C:\LTX2_3\ltx-2.3-22b-dev_audio_vae.safetensors
With Custom Clip-1 Model: C:\koboldcpp\gemma-3-12b-it-Q4_K_S.gguf
With Custom Clip-2 Model: C:\LTX2_3\ltx-2.3-22b-dev_embeddings_connectors.safetensors
Flash Attention is enabled
Offloading weights to system RAM

Swap to Diffusion Model Path:C:\LTX2_3\ltx-2.3-22b-dev-UD-Q5_K_S.gguf
Setting sd backend list to "", params backend list to "CPU"
model_path:
clip_l_path: C:\koboldcpp\gemma-3-12b-it-Q4_K_S.gguf
clip_g_path: C:\LTX2_3\ltx-2.3-22b-dev_embeddings_connectors.safetensors
clip_vision_path:
t5xxl_path:
llm_path:
llm_vision_path:
diffusion_model_path: C:\LTX2_3\ltx-2.3-22b-dev-UD-Q5_K_S.gguf
high_noise_diffusion_model_path:
embeddings_connectors_path:
vae_path: C:\LTX2_3\ltx-2.3-22b-dev_video_vae.safetensors
audio_vae_path: C:\LTX2_3\ltx-2.3-22b-dev_audio_vae.safetensors
taesd_path:
control_net_path:
photo_maker_path:
tensor_type_rules:
vae_decode_only: false
free_params_immediately: false
n_threads: 15
wtype: NONE
rng_type: cuda
sampler_rng_type: NONE
prediction: NONE
offload_params_to_cpu: true
max_vram: 0.000
backend:
params_backend: CPU
keep_clip_on_cpu: false
keep_control_net_on_cpu: false
keep_vae_on_cpu: false
flash_attn: true
diffusion_flash_attn: true
circular_x: false
circular_y: false
chroma_use_dit_mask: false
chroma_use_t5_mask: false
chroma_t5_mask_pad: 1


ggml_cuda_init: found 1 CUDA devices (Total VRAM: 48934 MiB):

  Device 0: NVIDIA RTX PRO 5000 Blackwell, compute capability 12.0, VMM: yes, VRAM: 48934 MiB

ggml was not compiled with any CUDA arch warning
Found 2 backend devices:
#0: CUDA0
#1: CPU
Initializing backend: CUDA0
Initializing backend: CPU
loading diffusion model from 'C:\LTX2_3\ltx-2.3-22b-dev-UD-Q5_K_S.gguf'
load C:\LTX2_3\ltx-2.3-22b-dev-UD-Q5_K_S.gguf using gguf format
init from 'C:\LTX2_3\ltx-2.3-22b-dev-UD-Q5_K_S.gguf'
loading llm from 'C:\koboldcpp\gemma-3-12b-it-Q4_K_S.gguf'
load C:\koboldcpp\gemma-3-12b-it-Q4_K_S.gguf using gguf format
init from 'C:\koboldcpp\gemma-3-12b-it-Q4_K_S.gguf'
loading vae from 'C:\LTX2_3\ltx-2.3-22b-dev_video_vae.safetensors'
load C:\LTX2_3\ltx-2.3-22b-dev_video_vae.safetensors using safetensors format
init from 'C:\LTX2_3\ltx-2.3-22b-dev_video_vae.safetensors', prefix = 'vae.'
loading embeddings connectors from 'C:\LTX2_3\ltx-2.3-22b-dev_embeddings_connectors.safetensors'
load C:\LTX2_3\ltx-2.3-22b-dev_embeddings_connectors.safetensors using safetensors format
init from 'C:\LTX2_3\ltx-2.3-22b-dev_embeddings_connectors.safetensors', prefix = ''
loading LTX audio VAE from 'C:\LTX2_3\ltx-2.3-22b-dev_audio_vae.safetensors'
load C:\LTX2_3\ltx-2.3-22b-dev_audio_vae.safetensors using safetensors format
init from 'C:\LTX2_3\ltx-2.3-22b-dev_audio_vae.safetensors', prefix = ''
Version: LTXAV
Weight type stat:                      f32: 2961 |    q8_0: 311  |    q4_K: 326  |    q5_K: 1240 |    q6_K: 204  |    bf16: 1531
Conditioner weight type stat:          f32: 289  |    q4_K: 326  |    q5_K: 10   |    q6_K: 1
Diffusion model weight type stat:      f32: 2672 |    q8_0: 311  |    q5_K: 1230 |    q6_K: 203  |    bf16: 28
VAE weight type stat:                 bf16: 272
ggml tensor size = 400 bytes
Try read vocab from C:\koboldcpp/embd_res/gemma2_vocab_json.embd

Try read vocab from C:\koboldcpp/embd_res/gemma2_merges_utf8_c_str.embd

vocab size: 262144
merges size 514905
llm: num_layers = 48, vocab_size = 262208, hidden_size = 3840, intermediate_size = 15360
using VAE for encoding / decoding
Using flash attention
Using flash attention in the diffusion model
loading weights
gemma3_12b params backend buffer size =  9661.05 MB(RAM) (626 tensors)
ltxav_text_projection params backend buffer size =  2205.02 MB(RAM) (4 tensors)
ltxav params backend buffer size =  15896.60 MB(RAM) (4444 tensors)
ltx_video_vae params backend buffer size =  1385.02 MB(RAM) (170 tensors)
ltx_audio_vae params backend buffer size =  339.88 MB(RAM) (1285 tensors)
NOT using mmap for 'C:\LTX2_3\ltx-2.3-22b-dev-UD-Q5_K_S.gguf' (mmap disabled by caller)
NOT using mmap for 'C:\koboldcpp\gemma-3-12b-it-Q4_K_S.gguf' (mmap disabled by caller)
NOT using mmap for 'C:\LTX2_3\ltx-2.3-22b-dev_video_vae.safetensors' (mmap disabled by caller)
NOT using mmap for 'C:\LTX2_3\ltx-2.3-22b-dev_embeddings_connectors.safetensors' (mmap disabled by caller)
NOT using mmap for 'C:\LTX2_3\ltx-2.3-22b-dev_audio_vae.safetensors' (mmap disabled by caller)
model files processing completed in 0.01s
using 15 threads for model loading
  |=================================>                | 4444/6573 - 2.98GB/s

  |======================================>           | 5070/6573 - 2.73GB/s

  |=======================================>          | 5240/6573 - 2.47GB/s

  |=======================================>          | 5244/6573 - 2.09GB/s

  |==================================================| 6573/6573 - 2.05GB/s

loading tensors completed, taking 12.20s (read: 7.40s, memcpy: 0.00s, convert: 0.63s, copy_to_backend: 0.00s)
finished loaded file
total params memory size = 29487.56MB (VRAM 0.00MB, RAM 29487.56MB): text_encoders 11866.07MB(RAM), diffusion_model 15896.60MB(RAM), vae 1724.89MB(RAM), controlnet 0.00MB(N/A), pmid 0.00MB(N/A)
running in Flux FLOW mode
Setting to Video Generation Mode!
Load Image Model OK: True
Embedded KoboldAI Lite loaded.
Embedded API docs loaded.
Embedded SDUI loaded.
Llama.cpp UI loaded.
======
Active Modules: ImageGeneration
Inactive Modules: TextGeneration VoiceRecognition MultimodalVision MultimodalAudio NetworkMultiplayer ApiKeyPassword WebSearchProxy TextToSpeech VectorEmbeddings AdminControl MCPBridge MusicGen RouterMode
Enabled APIs: KoboldCppApi A1111ForgeApi ComfyUiApi
Note: For third party Ollama API Emulation, you should set the port to 11434.
Starting Kobold API on port 5001 at http://localhost:5001/api/
Starting OpenAI Compatible API on port 5001 at http://localhost:5001/v1/
Starting llama.cpp secondary WebUI at http://localhost:5001/lcpp/
StableUI is available at http://localhost:5001/sdui/
======
Please connect to custom endpoint at http://localhost:5001
::1 - - [03/Jun/2026 08:43:45] "GET /sdapi/v1/sd-models HTTP/1.1" 200 -

Input: {"width": 1536, "height": 1024, "denoising_strength": 0.7, "frames": 129, "fps": 24, "enable_hr": false, "send_as_refimg": true, "reverse_refimg": false, "seed": 666, "cfg_scale": 5, "steps": 20, "sampler_name": "euler", "prompt": "a kobold roaching a marshmallow over a campfire", "negative_prompt": "static characters, camera pan", "init_images": [snipped], "inpainting_mask_invert": null, "inpainting_fill": null, "video_output_type": 2, "rep_pen": 1.0, "memory": "", "stop_sequence": []}

KCPP SD: Requested dimensions 1536x1024 changed to 1248x832

[08:43:45] Generating Image (20 steps)

resize input image from 1536x1024 to 1248x832

ImageGen References: RefImg=0 Wan=1 Photomaker=0

VID PROMPT:a kobold roaching a marshmallow over a campfire
NPROMPT:static characters, camera pan
CLPSKP:-1
SIZE:1248x832
STEP:20
SEED:666
STRENGTH:0.7
FRAMES:129
CTRL_FRM:0
INIT_IMGS:1


get_sigmas with LTX2 scheduler
LTX2 scheduler: tokens=17238, shift=6.7558, stretch=1, terminal=0.1000
sampling using Euler method
IMG2VID
VAE Tile size: 10x10
num tiles : 6, 4
optimal overlap : 0.420000, 0.466667 (targeting 0.500000)
processing 24 tiles
ltx_video_vae compute buffer size: 113.67 MB(VRAM)
  |==================================================| 24/24 - 5.03it/s

computing vae encode graph completed, taking 4.84s
encode_first_stage completed, taking 4842 ms
split prompt "a kobold roaching a marshmallow over a campfire" to tokens ["a", "Ôûükob", "old", "Ôûüro", "aching", "Ôûüa", "Ôûümarshmallow", "Ôûüover", "Ôûüa", "Ôûücampfire", ]
gemma3_12b compute buffer size: 2153.01 MB(VRAM)
gemma3_12b offload params (9661.10 MB, 626 tensors) to runtime backend (CUDA0), taking 1.30s
ltxav_text_projection compute buffer size: 15.96 MB(VRAM)
ltxav_text_projection offload params (2205.02 MB, 4 tensors) to runtime backend (CUDA0), taking 0.23s
computing LTXAV condition graph completed, taking 2432 ms
split prompt "static characters, camera pan" to tokens ["static", "Ôûücharacters", ",", "Ôûücamera", "Ôûüpan", ]
gemma3_12b compute buffer size: 2153.01 MB(VRAM)
gemma3_12b offload params (9661.10 MB, 626 tensors) to runtime backend (CUDA0), taking 1.09s
ltxav_text_projection compute buffer size: 8.71 MB(VRAM)
ltxav_text_projection offload params (2205.02 MB, 4 tensors) to runtime backend (CUDA0), taking 0.20s
computing LTXAV condition graph completed, taking 1875 ms
get_learned_condition completed, taking 4.31s
generate_video 1248x832x129
sample 39x26x17
ltxav compute buffer size: 11668.06 MB(VRAM)
ltxav offload params (15896.60 MB, 4444 tensors) to runtime backend (CUDA0), taking 2.33sC:\koboldcpp\ggml\src\ggml-cuda\cpy.cu:229: GGML_ASSERT(grid_z < USHRT_MAX) failed

[askmyteapot]

LostRuins@0ba7983
That was the change made to rectify the issue for the bug.

I should add, the Vulkan backend isn't impacted.

[CMay]

@Meet91721 @0cc4m @ggerganov

I'm also getting this issue as well with the ROCm backend of stable-diffusion.cpp when doing larger generations with LTX 2.3.

We're hitting these assert lines:
https://github.com/ggml-org/ggml/blame/master/src/ggml-cuda/cpy.cu#L229

I don't know about the proposed fix above as I'm not setup to compile from the source, but just know that interest exists in this section of code now if there is a safe workaround other than an assert. 😊

[ORippler]

@askmyteapot do you have a failing repro we can add to test-backend-ops?

[askmyteapot]

@askmyteapot do you have a failing repro we can add to test-backend-ops?

I don't sorry, as i used an LLM to help root-cause it and suggest fixes (which i tested and it did work).
My basic understanding is that we just need to add in a fallback rather than just stopping at that assert.

[CMay]

@ORippler I'll look at finding a minimum repro for my hardware and see if that's useful to you.

[CMay]

@ORippler Ok, did some testing and this seems to be close to the minimum I can repro it with, while avoiding other crashes and OOM scenarios, using the smallest necessary models with minimum settings to produce a result faster for testing. Since LTX 2.3 works without the audio vae and this produces the assert without the audio vae, I've excluded it to avoid an extra download. Even though it's a video model, I set it up to produce a single frame, since that's faster.

I'm using a Radeon 7900XTX with 24GB of VRAM using ROCm on Windows 11, but if you can reproduce it on an Nvidia GPU on some other platform, maybe that's useful too.

Grab the latest stable-diffusion.cpp:

• https://github.com/leejet/stable-diffusion.cpp/releases

Grab these LTX 2.3 model files:

• https://huggingface.co/unsloth/gemma-3-12b-it-GGUF/resolve/main/gemma-3-12b-it-UD-Q2_K_XL.gguf?download=true

• https://huggingface.co/unsloth/LTX-2.3-GGUF/resolve/main/distilled-1.1/ltx-2.3-22b-distilled-1.1-Q2_K.gguf?download=true

• https://huggingface.co/unsloth/LTX-2.3-GGUF/resolve/main/text_encoders/ltx-2.3-22b-distilled_embeddings_connectors.safetensors?download=true

• https://huggingface.co/unsloth/LTX-2.3-GGUF/resolve/main/vae/ltx-2.3-22b-distilled_video_vae.safetensors?download=true

• https://huggingface.co/Lightricks/LTX-2.3/resolve/main/ltx-2.3-spatial-upscaler-x2-1.1.safetensors?download=true

Run it exactly like this:

sd-cli -M vid_gen --diffusion-model "ltx-2.3-22b-distilled-1.1-UD-Q2_K.gguf" --vae "ltx-2.3-22b-distilled_video_vae.safetensors" --llm "gemma-3-12b-it-UD-Q2_K_XL.gguf" --embeddings-connectors "ltx-2.3-22b-distilled_embeddings_connectors.safetensors" -v --cfg-scale 1.0 --steps 1 --sampling-method euler_a --fa --temporal-tiling --extra-tiling-args "temporal_tile_frames=2,temporal_tile_overlap=1" --vae-on-cpu --vae-tiling --vae-tile-size 16x16 --hires --hires-steps 4 --hires-upscalers-dir "." --hires-upscaler "ltx-2.3-spatial-upscaler-x2-1.1" --hires-upscale-tile-size 64 -W 2144 -H 2144 --video-frames 1 --fps 1 -o ggml_gridassert_passed.webp --backend te=gpu,diffusion=gpu,firststage=cpu,autoencoder=gpu --params-backend disk -p "test prompt"

Approximate expected log:

[DEBUG] ggml_extend_backend.cpp:326  - Initializing backend: ROCm0
[DEBUG] model_loader.cpp:227  - using 6 threads for model loading
[INFO ] stable-diffusion.cpp:378  - loading diffusion model from 'ltx-2.3-22b-distilled-1.1-UD-Q2_K.gguf'
[INFO ] model_loader.cpp:235  - load ltx-2.3-22b-distilled-1.1-UD-Q2_K.gguf using gguf format
[DEBUG] model_loader.cpp:284  - init from 'ltx-2.3-22b-distilled-1.1-UD-Q2_K.gguf'
[INFO ] stable-diffusion.cpp:432  - loading llm from 'gemma-3-12b-it-UD-Q2_K_XL.gguf'
[INFO ] model_loader.cpp:235  - load gemma-3-12b-it-UD-Q2_K_XL.gguf using gguf format
[DEBUG] model_loader.cpp:284  - init from 'gemma-3-12b-it-UD-Q2_K_XL.gguf'
[INFO ] stable-diffusion.cpp:446  - loading vae from 'ltx-2.3-22b-distilled_video_vae.safetensors'
[INFO ] model_loader.cpp:238  - load ltx-2.3-22b-distilled_video_vae.safetensors using safetensors format
[DEBUG] model_loader.cpp:312  - init from 'ltx-2.3-22b-distilled_video_vae.safetensors', prefix = 'vae.'
[INFO ] stable-diffusion.cpp:463  - loading embeddings connectors from 'ltx-2.3-22b-distilled_embeddings_connectors.safetensors'
[INFO ] model_loader.cpp:238  - load ltx-2.3-22b-distilled_embeddings_connectors.safetensors using safetensors format
[DEBUG] model_loader.cpp:312  - init from 'ltx-2.3-22b-distilled_embeddings_connectors.safetensors', prefix = ''
[INFO ] stable-diffusion.cpp:497  - Version: LTXAV
[INFO ] stable-diffusion.cpp:526  - Weight type stat:                      f32: 2961 |    q8_0: 166  |    q2_K: 938  |    q3_K: 476  |    q4_K: 185  |    q5_K: 97   |    q6_K: 169  |  iq2_xs: 10   | iq3_xxs: 10   |   iq3_s: 15   |   iq2_s: 10   |  iq4_xs: 5    |    bf16: 202
[INFO ] stable-diffusion.cpp:527  - Conditioner weight type stat:          f32: 289  |    q2_K: 152  |    q3_K: 129  |    q4_K: 5    |    q6_K: 1    |  iq2_xs: 10   | iq3_xxs: 10   |   iq3_s: 15   |   iq2_s: 10   |  iq4_xs: 5
[INFO ] stable-diffusion.cpp:528  - Diffusion model weight type stat:      f32: 2672 |    q8_0: 166  |    q2_K: 786  |    q3_K: 347  |    q4_K: 180  |    q5_K: 97   |    q6_K: 168  |    bf16: 28
[INFO ] stable-diffusion.cpp:529  - VAE weight type stat:                 bf16: 170
[DEBUG] stable-diffusion.cpp:531  - ggml tensor size = 400 bytes
[DEBUG] gemma_tokenizer.cpp:32   - vocab size: 262144
[DEBUG] gemma_tokenizer.cpp:40   - merges size 514905
[DEBUG] llm.hpp:226  - llm: num_layers = 48, vocab_size = 262208, hidden_size = 3840, intermediate_size = 15360
[DEBUG] ltxv.hpp:265  - ltxav: num_layers = 48, hidden_size = 4096, num_attention_heads = 32, audio_hidden_size = 2048, audio_num_attention_heads = 32
[INFO ] stable-diffusion.cpp:930  - using VAE for encoding / decoding
[INFO ] stable-diffusion.cpp:1026 - Using flash attention
[INFO ] stable-diffusion.cpp:1040 - Using flash attention in the diffusion model
[DEBUG] stable-diffusion.cpp:1058 - validating model metadata
[DEBUG] stable-diffusion.cpp:1108 - model metadata validated; weights will be prepared lazily
[INFO ] stable-diffusion.cpp:1163 - total params memory size = 22479.72MB (VRAM 22479.72MB, RAM 0.00MB): text_encoders 9887.39MB(VRAM), diffusion_model 11207.31MB(VRAM), vae 1385.02MB(VRAM), controlnet 0.00MB(N/A), extensions 0.00MB(N/A)
[INFO ] stable-diffusion.cpp:1254 - running in Flux FLOW mode
[INFO ] denoiser.hpp:623  - get_sigmas with LTX2 scheduler
[DEBUG] denoiser.hpp:536  - LTX2 scheduler: tokens=4489, shift=2.1907, stretch=1, terminal=0.1000
[INFO ] stable-diffusion.cpp:3397 - sampling using Euler A method
[DEBUG] bpe_tokenizer.cpp:207  - split prompt "test prompt" to tokens ["test", "▁prompt", ]
[DEBUG] model_loader.cpp:990  - loading 626/626 tensors from gemma-3-12b-it-UD-Q2_K_XL.gguf
  |##################################################| 626/626 - 1.43GB/s
[INFO ] model_loader.cpp:1229 - loading tensors completed, taking 3.17s (read: 1.20s, memcpy: 0.00s, convert: 0.09s, copy_to_backend: 0.24s)
[DEBUG] model_manager.cpp:221  - model manager prepared params backend buffer (7682.39 MB, 626 tensors, VRAM)
[DEBUG] ggml_extend.hpp:1998 - gemma3_12b compute buffer size: 2153.01 MB(VRAM)
[DEBUG] model_loader.cpp:990  - loading 4/4 tensors from ltx-2.3-22b-distilled_embeddings_connectors.safetensors
  |##################################################| 4/4 - 1.23GB/s
[INFO ] model_loader.cpp:1229 - loading tensors completed, taking 1.76s (read: 0.50s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.05s)
[DEBUG] model_manager.cpp:221  - model manager prepared params backend buffer (2205.02 MB, 4 tensors, VRAM)
[DEBUG] ggml_extend.hpp:1998 - ltxav_text_projection compute buffer size: 4.35 MB(VRAM)
[DEBUG] conditioner.hpp:2297 - computing LTXAV condition graph completed, taking 7511 ms
[INFO ] stable-diffusion.cpp:4876 - get_learned_condition completed, taking 7.51s
[DEBUG] model_manager.cpp:767  - model manager releasing params backend buffer (7682.39 MB, 626 tensors, VRAM)
[DEBUG] model_manager.cpp:767  - model manager releasing params backend buffer (2205.02 MB, 4 tensors, VRAM)
[INFO ] stable-diffusion.cpp:5178 - generate_video 2144x2144x1 -> LTX latent spatial upscale
[DEBUG] stable-diffusion.cpp:5237 - sample 67x67x1
[DEBUG] model_loader.cpp:990  - loading 4444/4444 tensors from ltx-2.3-22b-distilled-1.1-UD-Q2_K.gguf
  |##################################################| 4444/4444 - 2.63GB/s
[INFO ] model_loader.cpp:1229 - loading tensors completed, taking 3.86s (read: 2.85s, memcpy: 0.00s, convert: 0.04s, copy_to_backend: 0.66s)
[DEBUG] model_manager.cpp:221  - model manager prepared params backend buffer (11207.31 MB, 4444 tensors, VRAM)
[DEBUG] ggml_extend.hpp:1998 - ltxav compute buffer size: 910.94 MB(VRAM)
  |==================================================| 1/1 - 9.22s/it
[DEBUG] model_manager.cpp:767  - model manager releasing params backend buffer (11207.31 MB, 4444 tensors, VRAM)
[INFO ] stable-diffusion.cpp:5270 - sampling completed, taking 9.43s
[INFO ] stable-diffusion.cpp:4960 - LTX latent spatial upscale: latent 67x67x1x128 -> model output
[DEBUG] model_loader.cpp:990  - loading 2/170 tensors from ltx-2.3-22b-distilled_video_vae.safetensors
  |##################################################| 2/2 - 0.00MB/s
[INFO ] model_loader.cpp:1229 - loading tensors completed, taking 0.11s (read: 0.00s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.06s)
[DEBUG] model_manager.cpp:221  - model manager prepared params backend buffer (  0.00 MB, 2 tensors, VRAM)
[DEBUG] ggml_extend.hpp:1998 - ltx_video_vae compute buffer size: 2.19 MB(VRAM)
[DEBUG] model_loader.cpp:227  - using 6 threads for model loading
[INFO ] model_loader.cpp:238  - load ltx-2.3-spatial-upscaler-x2-1.1.safetensors using safetensors format
[DEBUG] model_loader.cpp:312  - init from 'ltx-2.3-spatial-upscaler-x2-1.1.safetensors', prefix = ''
[DEBUG] ltx_latent_upscaler.hpp:124  - ltx latent upsampler: in_channels = 128, mid_channels = 1024, num_blocks_per_stage = 4, spatial_scale = 2.000, temporal_up_factor = 1, rational_resampler = 0
[DEBUG] model_loader.cpp:990  - loading 72/72 tensors from ltx-2.3-spatial-upscaler-x2-1.1.safetensors
  |##################################################| 72/72 - 1.35GB/s
[INFO ] model_loader.cpp:1229 - loading tensors completed, taking 0.69s (read: 0.22s, memcpy: 0.00s, convert: 0.27s, copy_to_backend: 0.05s)
[DEBUG] model_manager.cpp:221  - model manager prepared params backend buffer (949.72 MB, 72 tensors, VRAM)
[DEBUG] ggml_extend.hpp:1998 - ltx_latent_upsampler compute buffer size: 1159.51 MB(VRAM)
[DEBUG] model_manager.cpp:767  - model manager releasing params backend buffer (949.72 MB, 72 tensors, VRAM)
[INFO ] stable-diffusion.cpp:5283 - LTX latent spatial upscale completed, taking 1.23s
[INFO ] denoiser.hpp:623  - get_sigmas with LTX2 scheduler
[DEBUG] denoiser.hpp:536  - LTX2 scheduler: tokens=17956, shift=7.0129, stretch=1, terminal=0.1000
[DEBUG] stable-diffusion.cpp:5360 - sample(latent upscale) 134x134x1
[INFO ] stable-diffusion.cpp:5366 - LTX latent spatial upscale refine: scheduler_steps=5, denoising_strength=0.70, sampler=Euler A, sigma_sched_size=5
[DEBUG] model_loader.cpp:990  - loading 4444/4444 tensors from ltx-2.3-22b-distilled-1.1-UD-Q2_K.gguf
  |##################################################| 4444/4444 - 2.58GB/s
[INFO ] model_loader.cpp:1229 - loading tensors completed, taking 3.93s (read: 3.00s, memcpy: 0.00s, convert: 0.04s, copy_to_backend: 0.62s)
[DEBUG] model_manager.cpp:221  - model manager prepared params backend buffer (11207.31 MB, 4444 tensors, VRAM)
[DEBUG] ggml_extend.hpp:1998 - ltxav compute buffer size: 3449.15 MB(VRAM)
D:/a/stable-diffusion.cpp/stable-diffusion.cpp/ggml/src/ggml-cuda/cpy.cu:229: GGML_ASSERT(grid_z < USHRT_MAX) failed

[ORippler]

@CMay I meant something like 46a61e1, but thanks for your effort regardless 😃

Feel free to try out #25000. Note that the fall-through will be slower, if you can show some traces/timings that identify this as a bottle-neck I can take another look