Best way to run DS4 Flash on 4xR9700?
So, I finally maxed out my system - 4 R9700s and my old Xeon 8368 for a total of 128GB VRAM and 256GB of 8-channel DDR4. However... DS4 Flash barely runs fast now compared to the 3 card version. I'm using llama.cpp and standard fitting with a 200k context, using the bartowski mxfp4 quant. Is there a better way to do it? Based on the Unsloth eval charts, it seems like picking a smaller variant that fits fully in 128GB is going to give up a lot of quality.
评论
?
参与讨论