Getting slower speeds WITH MTP on Gemma 4 12B QAT than without...

Hey, wondering if anyone's seen this issue themselves? I'm using a 16gb 9060XT on a proxmox LXC, llama-server via docker on the Vulkan backend, and it's been serving me fantastically - 40-50t/s on most modesl with MTP, even 25t/s with the IQ2 or IQ3 of 3.8 27B with around 100k ctx! But I've been experimenting with Gemma 4 12B QAT and I'm noticing something odd: despite being only ca~ 12gb in VRAM with 262k context and vision enabled, I get around 33t/s though I would expect this to be higher considering its size. Even weirder is if I add MTP - VRAM usage goes up to 12.5gb or so, but the t/s goes DOWN. At draft n max = 1 I get 32 t/s, dropping to 23t/s by draft n max = 4. I've tried reducing context, disabling vision - hell I've even tried multiple repos of the QAT including Unsloths and HuahuaCS. Any ideas what could be happening? If it helps, here's my launch commands for the docker stack: command: > --models-preset /models/models.ini --models-max 1 --timeout 28800 --port 8080 --host 0.0.0.0 --no-mmap --metrics --kv-unified --jinja --sleep-idle-seconds 900 And from my models.ini: #version = 1 [*] flash-attn = on ngl = 99 t = 6 tb = 12 b = 2048 ub = 512 #cache-ram = 2048 reasoning = on reasoning-budget = 4000 reasoning-budget-message = "\n\n[SYSTEM: STOP REASONING. TIME TO RESPOND.]" reasoning-preserve = on [Gemma-4-12B] hf-repo = HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced:Q4_K_M temp = 0.6 top-p = 0.9 min-p = 0.05 top-k = 64 c = 262144 np = 1 repeat-penalty = 1.1 #b = 512 #spec-type = draft-mtp #spec-draft-n-max = 4 #spec-draft-p-min = 0.8 EDIT: Thanks to those who commented. I tried the things you suggested (as noted in my replies) and didn't see any improvement sadly. However, I decided on a whim to try ROCm rather than Vulkan, no other setting changes and suddenly: 60t/s with MTP, 30t/s without. So, something about the current server-vulkan image is killing MTP on Gemma 4. I'll check out my other models I use to ensure ROCm isn't going to hamstring those just for the sake of faster Gemma-4-12B, but at least I'm in a position now where I can use this model and its MTP heads! Thanks once again <3