Running UNCENSORED DeepSeek-V4.1-Flash-1017 on a 12 Gb VRAM

Measured, refusals 85%→0% even in thinking, 3 sizes 12 GB GPU VRAM, 64GB RAM + SSD via Strata Engine We took ISTA-DASLab's GSQ-RCO quants of DeepSeek-V4.1-Flash-1017 (the ones people run in Strata with a 12 GB GPU and the Engram table on SSD) and removed the refusals. We only touched 82 dense tensors per file: attention output, shared expert down proj and the Engram value proj. The 384 routed experts are byte for byte the same as ISTA's, so their GSQ rounding is kept. Headers and file sizes match too, so you just swap the shards and keep using ISTA's Engram file. Refusals are on JailbreakBench, stock rung -> ours, same Strata setup: Rung Size (+110 GB Engram) Refusals Refusals in thinking (of 30) MMLU-500 KL vs stock IQ1_M 121 GB 88.9% -> 0% 27 -> 1 80.6% 0.021 IQ2_XXS 138 GB 85.2% -> 0% 27 -> 0 82.8% 0.023 IQ2_XS 155 GB 86.4% -> 0% 27 -> 2 81.6% 0.027 We also ran GPQA-Diamond and AIME25 on IQ2_XXS against stock to make sure it didn't get dumber. GPQA came out 83.3% vs 83.3%, AIME 80.0% vs 76.7% (only 30 problems, so call it the same). Things to know: IQ2_XS still refuses inside its thinking on 2 of 30 prompts. The other two are at 0 and 1, so I'd grab IQ2_XXS. We tested at 8K context on a rented GPU. Haven't checked long context or speed on a 12 GB card ourselves, so if you try it please post your tok/s and RAM. Strata only for now. If you're on llama.cpp, our older release with the same edit works there: huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GGUF Model: huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF All the measurements: huggingface.co/datasets/AtomicChat/Deep...4.1-Flash-1017-Uncensored-GGUF-metrics Strata Engine: github.com/niko1221/strata Thanks to DeepSeek for the base model (MIT), ISTA-DASLab for GSQ-RCO and the Engram file, and the Strata dev for the engine. I'm on the Atomic.Chat team that made this.

参与讨论