I turned a 12 GB RTX 3060 into a Qwen API server for 4 people at once
Last time it was "Strata for all Qwen models". This time: share it. strata serve now runs a 35B MoE (Qwen3.6-35B-A3B Q2_K_XL) on one RTX 3060 for several users at the same time: Users at once Total Each user 1 106 tok/s 106 tok/s 2 139 tok/s 69 tok/s 4 164 tok/s 41 tok/s 41 tok/s each is still faster than most people read. What's in it: OpenAI and Anthropic APIs , so Open WebUI, OpenCode and even Claude Code connect to it. API keys from a file , reloaded when you edit it, plus a usage log per key. That's enough to give access to friends, a team, or paying users. Fair queue : a free slot goes to the key with the fewest requests running, and --max-per-key stops one user from taking everything. /health and Prometheus /metrics for load balancers and dashboards. NVIDIA, AMD and Intel (CUDA or Vulkan). Two cards can split one model ( --device 0,1 ). One binary, written in Rust. No Python, no Docker. strata serve --model Qwen3.6-35B-A3B-UD-Q2_K_XL.gguf --gpu --parallel 4 --api-keys keys.txt Setup guide: docs/HOSTING.md. Repo: RustingStrata What would you host on it?