Is there a better small model than Qwen3.5 4B for a fast local AI assistant?

I'm currently using Qwen3.5 4B as the brain of my local AI assistant because my hardware is relatively limited. One thing I really like about it is the speed. On my system, I'm getting around 40–50 tokens/sec, which makes the interaction feel surprisingly close to real-time. So I don't want to move to a much larger model if that means losing that responsiveness. What I'm wondering is whether there are any newer or alternative models in roughly the same size/memory/compute range that are noticeably better than Qwen3.5 4B for my particular use case. My priorities are mainly: - General-purpose conversation and communication - Natural responses and understanding - Reasoning and following instructions - Multilingual conversations and translation - Context understanding - Tool/function calling mainly for triggering actions in my assistant, rather than writing code - Reliability when interacting with tools - Ideally good performance while keeping the model fast and lightweight Essentially, I'm trying to build a local assistant where the model acts as the reasoning/conversation layer, while other tools handle things like actions, APIs, automation, etc. So my question is: If I'm already using Qwen3.5 4B and getting ~40 to 50 TPS, is there anything in a similar practical memory/compute footprint that you'd personally consider a significant upgrade for general conversation + multilingual ability + tool calling? It doesn't necessarily have to have fewer parameters. I'm more interested in overall capability relative to the amount of RAM/VRAM and compute it requires. I'm happy with Qwen3.5 4B, so I'm not looking for something that's only marginally different. I'm wondering if there's something that makes you think

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论