Up to 3.2x Faster Inference with LFM2.5-DSpark

localllama (Reddit)，r/LocalLLaMA，约 72.8 万成员的社区，专注于在本地硬件上运行大语言模型，涵盖模型选择、GPU 配置、量化技术、Ollama 与 llama.cpp 等工具，以及隐私优先的 AI 工作流。关注

This one is ready to use now onwards, as its PR got merged today mentioned by u/jacek2023 But don't use DSpark GGUFs(testing versions) from that PR. Use the official GGUFs by them. You could find them on model cards. Anyway sharing the table below. Draft (GGUF) Target (GGUF) LFM2.5-1.2B-Instruct-DSpark-GGUF LFM2.5-1.2B-Instruct-GGUF LFM2.5-2.6B-DSpark-GGUF LFM2.5-2.6B-GGUF LFM2.5-8B-A1B-DSpark-GGUF LFM2.5-8B-A1B-GGUF --------- Never tried speculative decoding on Mobile. I use PocketPal & ChatterUI. Any idea how to run these on Mobile?

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论