Two days tuning a single Tesla P40: 35B MoE, full 262k context, ~60 t/s short and ~30 t/s at 100k deep. No sub-Q4 quants.

i Got a bit obsessed with my P40 this week trying to max speed on Ornith 1.5 35B-A3B (qwen3.5 MoE arch) without going below Q4 weights or q8 KV. Shrinking quants to win a benchmark has not really been useful to me so ai wanted a logical middle ground that was still “fast” Setup: one Tesla P40 24GB on PCIe x1 (yes, x1), Windows, my own llama.cpp fork. Q4_0 built from the official BF16, MTP draft (n=2), full 262k context, q8_0 KV. What actually moved it: - ECC off + 1.5 GB VRAM and ~9% decode. - q8_0 KV, not q4_0. On Pascal q4 KV falls apart with depth (30.6 vs 42.2 t/s at 32k). - Check your MTP head. The first upload of this model, and quants built from it, have an untrained MTP head. blk.40.nextn.shared_head_norm mean ~1.02 = untrained, ~2.93 = trained. Draft acceptance went from ~45% to 60-85%. - int8 dp4a for the KQ half of the flash-attention tile kernel. Pascal has no tensor cores and crippled fp16, but dp4a is fast. Prefill +26% at 32k, +41% at 100k. - MTP prefill window. The MTP head is a full-attention layer, so on long prompts it costs ~13% prefill. Letting it read only the last 8k of the prompt gets that back (121k prompt: 327 -> 373 t/s). Output doesn't change, the main model still verifies every draft. Didn't work: f16 KV (faster in llama-bench, but slower and less context once MTP is on in a real server), dequantizing V inside the kernel (-5%), the official Q4_K_M (same quality score as my Q4_0, slower, less context). llama-bench, no MTP, 2 runs each: depth prefill t/s decode t/s 0 1104 60.8 8k 880 57.0 32k 552 46.0 64k 366 35.8 100k 265 29.1 190k - 18.8 Real server with MTP: 65-74 t/s on short prompts. A 101k-token prompt reads at ~415 t/s and generates at 35-39 t/s. I couldn't find anyone else posting P40 numbers at depth, so hopefully the curve helps someone. Happy to share the patches if you want to try them on your own Pascal card.