Homelab AI Expansion Thoughts?
Contemplating expanding my homelab AI solution, and would like to get some input from others on my planning thoughts. Currently running a dedicated box with below.
Ubuntu 26.04 LTS
5070 16GB VRAM
64GB RAM
10G LAN
Use cases are Ollama, Open-WebUI, Hermes, code assistance, and a bunch of automation tools like n8n that use ollama externally. Can’t do video generation or some of the better and larger models which is a big blocker right now.
The issue is pretty clear, not enough VRAM. At this point, I have 4 solutions with different costs:
- Buy a used Bitcoin mining frame, get a nice server mobo that has 4-8 PCIE x16 slots and scale out GPU’s as wanted and based on price
- 2 x NVIDIA DGX Sparks with direct connection in a cluster
- 2 x AMD Halo with direct connection in a cluster
- 2 x Minisforum boxes
Would love to understand other peoples experience and appreciate any lessons learned. Thanks.
评论
?
参与讨论