Homelab AI Expansion Thoughts?

Contemplating expanding my homelab AI solution, and would like to get some input from others on my planning thoughts. Currently running a dedicated box with below.

Ubuntu 26.04 LTS
5070 16GB VRAM
64GB RAM
10G LAN
Use cases are Ollama, Open-WebUI, Hermes, code assistance, and a bunch of automation tools like n8n that use ollama externally. Can’t do video generation or some of the better and larger models which is a big blocker right now.

The issue is pretty clear, not enough VRAM. At this point, I have 4 solutions with different costs:

  1. Buy a used Bitcoin mining frame, get a nice server mobo that has 4-8 PCIE x16 slots and scale out GPU’s as wanted and based on price
  2. 2 x NVIDIA DGX Sparks with direct connection in a cluster
  3. 2 x AMD Halo with direct connection in a cluster
  4. 2 x Minisforum boxes

Would love to understand other peoples experience and appreciate any lessons learned. Thanks.

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论