LFM2.5 2.6b vs MiniCPM5 2b

I tried both these models on a few small agentic tasks with tools (i.e. "What's the weather like today?", etc.). They're both pretty solid at using web search to find answers despite being small models. TLDR: LFM2.5 2.6b is the clear winner. Somehow it's faster and uses less ram than MiniCPM despite having more parameters. It also seems better aligned for english conversation. MiniCPM5 On an M1 air: pp 162 t/s - tg 16 t/s Uses about 3.8gb of ram at 32k context (with draft model) Often responds in chinese despite my prompting in english. It's very smart when it does respond in english and may be stronger at agentic work. It uses more memory than LFM2.5 despite supposedly having less parameters. There's a corresponding dspark model available. ​ llama-server --model MiniCPM5-2B-Q8_0.gguf -md MiniCPM5-2B-DSpark-Q8_0.gguf --load-mode none --spec-type draft-dspark --spec-draft-n-max 2 -ngl all -ngld all -fa on -np 1 -t 4 -c 32000 --reasoning on -fit off --temp 1.0 --top-p 0.95 --cache-type-k q5_1 --cache-type-v q5_1 --spec-draft-type-k q8_0 --spec-draft-type-v q8_0 LFM2.5 On an M1 air: pp 200 t/s - tg 22 t/s Uses about 2.5gb of ram at 32k context Works well for simple one shot agentic work but tends to start hallucinating quickly as the conversation gets longer. ​ llama-server -m LFM2.5-2.6B-QAD-Q4_0.gguf -ngl all -fa on --load-mode none --temp 0.1 --top-k 50 --top-p 0.9 -c 32000 --threads 4 --reasoning on -fit off --reasoning-preserve

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论