Can I self-host local AI (LLMs + image generation) alongside Jellyfin and Immich on my hardware?
Hi everyone! I'm planning to set up a local AI environment on my TrueNAS SCALE server and would appreciate advice from people who are already self-hosting AI. My current hardware: CPU: AMD Ryzen 5 3600 RAM: 32 GB GPU currently installed: Intel Arc A750 8 GB VRAM Spare GPU: Gigabyte NVIDIA RTX 2070 Super WINDFORCE OC 8 GB PSU: Cooler Master V850 Gold V2 850W 80+ Gold OS: TrueNAS SCALE Deployment: Docker Compose / Dockge My motherboard can accommodate only one GPU, so I need to choose between the Arc A750 and RTX 2070 Super. What I'd like to run: Local LLMs: Coding assistants and general-purpose chat models using LocalAI. I'm currently testing Qwen2.5-Coder 7B Q4_K_M. Local image generation: ComfyUI with Stable Diffusion models, including community checkpoints with fewer content restrictions. Jellyfin: Hardware-accelerated media transcoding. Immich: Machine learning and photo management. Ideally, I'd like these services to coexist on the same server, although I'm fine with stopping the LLM or image generator when I need more GPU resources. My questions: Would the RTX 2070 Super be a better choice than the Arc A750 for this workload, especially for CUDA compatibility? What are the best LLMs I can realistically run with 8 GB VRAM? Would 14B models be usable with partial CPU offloading? Can I run ComfyUI and generate images at reasonable speeds on the RTX 2070 Super? Is 32 GB system RAM sufficient, or would upgrading to 64 GB make a meaningful difference? Can Jellyfin, Immich and local AI share the same NVIDIA GPU reliably in Docker on TrueNAS SCALE? Would you recommend a different setup or workflow for managing GPU memory between these services? I'm not expecting to run huge models. I mainly want a practical, reasonably fast self-hosted setup for coding assistance, image generation and my existing media services. I'd appreciate real-world experiences, model recommendations and any tips for configuring this on TrueNAS SCALE. Thanks!