What are you guys running on an M5 Pro 64gb?
Is it all qwen3.8 27b? I get around 20 tok/s sustained, with like 30 tok/s in the first 2-4k tokens. It's still quite unbearably slow, maybe because it thinks so much, I've also tried antirez's DS4 and while it works I get around 8 tok/s sustained with deepseekv4 flash, which is so slow and basically unusable.
评论
?
参与讨论