llama.cpp cublas error, how to uninstall/reinstall properly (Linux Mint)

I am pretty dumb in this kinda stuff so please don't blame me for it. My llama serve kept crashing with Cublas errors on first prompt with some models, but my VRAM usage was 3000MB/8k. I chatted with sonnet 5.5 for a bit and it told me it was a CUDA version issue (and made it work by not selecting any CUDA device)... I don't know if this makes sense, please tell me if it doesn't/what problem you think there is. I realized that the best way was to delete llama.cpp (installed with curl install script) and install a clean CUDA 12 Ubuntu version. - How do I properly uninstall llama.cpp (I don't wanna mess with Ollama files)? - How do I install new version from .tar.gz archive without messing with system packages? -Does my error diagnosis make sense to you? Would CUDA make generation actually faster? (I am getting 8tk/s with Qwen35B Q2 on 4060Ti 8GB due to no CUDA selected)