70 benchmarked Open Source"Decision Models" (zero-shot classification)

huggingface.co/spaces/multimodalart/jev-decision-index Not surprising to see Qwen 3.8 27b, Gemma 31b and Gemma 26b A4B in the lead. Jev is the control all the way to the left. The top 3 aren't quantized, full BF16 precision so I haven't tested a fair few of them, but I have been able to test Winnow-12 which is 9th place here, based on Gemma 12b at Q8 it's pretty good, 32-50 ms for responses is lightning fast. It's 10th place if including Jev as #1 but out of open source options it's 9th. Anyone wanna cook up some quants for the top 3 repos?

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论