GLM-5.3-Flash Benchmarks on TensorSharp and llama.cpp

Model: unsloth/GLM-5.3-Flash-GGUF UD-Q2_K_XL (4 shards, 101 GiB) + mmproj-BF16. Reference: llama.cpp PR #27754 (glm5next), CUDA 12.8, SM 120. Throughput Both engines back to back in one session, flash attention on, n_ubatch 2048 on both ( llama-bench vs the parity harness --bench ). Run-to-run spread on throughput numbers is roughly 3–4%. test llama.cpp TensorSharp ratio pp2048 2070 t/s 2014 t/s 0.97× pp16384 1690 t/s 1692 t/s 1.00× pp32768 1483 t/s 1446 t/s 0.98× tg64 36.6 t/s 73.5 t/s 2.01× Decode at depth holds up: 40.9 t/s after a 17.7K-token prompt, 28.2 t/s after 36K. Weight load is 98.6 GiB across two GPUs in 17 s from a warm cache. CPU-MoE offload (first 10 layers’ experts host-resident) decodes at 35–40 t/s. Why decode is 2× The executor submits one ggml graph per ubatch through ggml_backend_sched with a shape-keyed LRU graph cache, so steady-state decode replays one allocated (and CUDA-captured) graph instead of rebuilding per token. The hybrid stack rewards this: 34 KDA layers plus hyper-connections make the per-token graph deep in small ops, which is exactly where per-token rebuild and launch overhead hurts most. Prefill has no such lever — both engines are GEMM-bound there — and lands within a few percent.