Is there a GLM 5.3 Flash Antirez/DS4 GGUF targeted at 192 GB RAM?
Is one possible? Can I make one? Antirez has a Q2 @ 96.5 GB which will lose quality compared to Q4 and wastes ~95 GB of my Mac’s RAM, and a Q4 @ 191 GB which is too big as it doesn’t leave enough room for OS let alone KV cache. I noticed that other Q4 and even Q4/Q8 mix GGUFs and MLXs not intended for DS4 range from 150 to 180 GB. Something that size and quality would be great for me, but I really want to use DS4 if I can. Can anyone offer any help? Thank you!
评论
?
参与讨论