How fast can I get a voice assistant to respond without a GPU? Qwen3-ASR and Kokoro-TTS ONNX on CPU.

Been testing out the ONNX models to see how far I can push the CPU to take on ASR and TTS, so the GPU is completely free for running the LLM. The video attached shows me testing latency on a 2022 Macbook M2 and an AMD Ryzen 9 7900. This is just running the regex fast commands, so most of the latency (apart from grabbing the spotify music) should be from the ASR and TTS. These ONNX models are really great. The M2 is mostly usable, the Ryzen 9 is blazing fast. The two models I am running are: Daumee/Qwen3-ASR

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论