What TPS is too slow for you?

localllama (Reddit)，r/LocalLLaMA，约 72.8 万成员的社区，专注于在本地硬件上运行大语言模型，涵盖模型选择、GPU 配置、量化技术、Ollama 与 llama.cpp 等工具，以及隐私优先的 AI 工作流。关注

I keep reading about extreme ends of TPS what people consider running their models at. Some are happy to run huge quality models at merely sub 30 tps, while others sacrifice/compromise the quality and pursue absolutely spectacular tps, often above 100-150. Where do you draw the line? Whats absolute minimum tps you will accept running local model at?

添加评论

点踩

分享查看原文

参与讨论