~22% less weight VRAM, lossless: base-3 packing for ternary GGUFs

I built a denser GGUF format for ternary models: Q2_B3 / “B3S” If you're running a ternary model like BitNet-b1.58 or Ternary-Bonsai, the weights are already restricted to -1, 0, or +1 times a block scale. That means a normal Q2 representation is leaving some space on the table. B3S packs the three possible weight values directly in base 3. With 128 weights per block, it's 26 bytes of packed trits + one f16 scale = 28 bytes/block, or 1.75 bits per weight. Rough weight sizes : 9B: ~2.5 GB Q2_0 → ~2.0 GB B3S 27B: ~7.6 GB Q2_0 → ~5.9 GB B3S That's weights only. Context/KV is separate, so figure another ~1–2 GB depending on what you're running. The important caveat: this is NOT a general 2-bit quantizer. If you feed it a normal FP16 model, quality will fall apart. The whole thing only makes sense when the source weights are already ternary. For a genuinely ternary model, the packing itself doesn't throw away another level of precision. You're still storing the same {-1, 0, +1} states and an f16 block scale, just using base-3 packing instead of a general-purpose 2-bit representation. The implementation is a fairly small llama.cpp fork based on commit 4e97ac86e. It adds the Q2_B3 type and the backend support around it. Backend status : AMD ROCm/HIP: this is the main path. Built and tuned on RDNA3/gfx1100, specifically a 7900 XTX. CPU: works. NVIDIA CUDA: compiles, but I don't own NVIDIA hardware, so I haven't verified it on-device. Apple Metal: same situation. Code is there and compiles, but I can't personally test it. So CUDA and Metal should be considered unverified for now. I don't have speed or perplexity tables yet either. Benchmarks done on my hardware show no appreciable loss of PPS or decoding speed There's also a separate repacker for older Q2_B3 GGUFs that use the 30-byte/two-scale block layout. It converts them to the current 28-byte/single-scale B3S layout. The repacker checks every block before doing that. If the second scale isn't actually redundant and removing it would change the weights, it aborts instead of silently producing a lossy file. Once you have a B3S GGUF, you run it normally with llama-cli from the fork. More implementation/format details are in README_B3S.md. If anyone here is running gfx1100, I'd be interested in independent results. More importantly, if someone has an NVIDIA or Apple machine and can compare CUDA/Metal output against a CPU run, that's probably the most useful testing gap right now. Fork : github.com/llopresto87/llama-cpp-ternary-b3s Repacker : github.com/llopresto87/ternary-q2_0-repacker Note : Posting this on behalf of u/llopresto87's request. He'll reply for your comments.