Dual RTX 3090 Qwen3.8-27B Help

I'm new to local LLMs and wondering if my performance looks normal or if I'm doing something wrong. My use case is local agentic coding. Build OS: Windows 10 - NO WSL CPU: AMD Ryzen 5 5600X Motherboard: ASRock X570 Taichi GPU: 2× NVIDIA RTX 3090 24GB ( 48GB total VRAM ) RAM: ~80GB llama.cpp: 0.1.2-dev Build: 10502 Commit: 0adcc3bb5 Compiler: Clang 20.1.8 for Windows x86_64 Model Qwen3.8-27B-UD-Q6_K_XL.gguf latest dynamic v3 from unsloth I'm generally seeing around 50–65 tokens/sec generation , sometimes dropping into the 40s. Current GPU Usage With Model Loaded NVIDIA-SMI 581.57 Driver Version: 581.57 CUDA Version: 13.0 GPU 0: NVIDIA GeForce RTX 3090 VRAM: 19841 MiB / 24576 MiB Driver Model: WDDM Display: On GPU 1: NVIDIA GeForce RTX 3090 VRAM: 21859 MiB / 24576 MiB Driver Model: WDDM Display: Off Exact llama-server Command llama-server -m "C:\Models\Qwen3.8\Qwen3.8-27B-UD-Q6_K_XL.gguf" --mmproj "C:\Models\Qwen3.8\mmproj-BF16.gguf" --alias Qwen3.8-27B --jinja --reasoning-format deepseek --n-gpu-layers all --split-mode layer --tensor-split 1,1 --ctx-size 262144 --parallel 1 --kv-unified --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --spec-type draft-mtp --spec-draft-n-max 3 --batch-size 2048 --ubatch-size 512 --reasoning auto --reasoning-effort medium --reasoning-preserve --reasoning-budget 16384 --reasoning-budget-message "Time to stop thinking. Give the final answer or make the tool call now." --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0 --repeat-penalty 1.0 --host 127.0.0.1 ` --port 8080 Any advice is appreciated.

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论