Deploying DeepSeek R1 Reasoning LLM Using SGLang

DeepSeek R1 is a first-generation reasoning model tuned for math, coding, and logical reasoning — reinforcement learning with a cold-start phase for readability and coherence, minimizing repetition and language mixing. This guide deploys it via SGLang in a ROCm-supported container on an AMD Instinct MI300X GPU server, then verifies inference over HTTP.

Prerequisites: access to an AMD Instinct MI300X GPU instance (large VRAM is required for this model's size).

Deploy DeepSeek R1

1. Install the Hugging Face CLI and start the model download in the background — it's large, so kick it off early and continue with the next steps while it completes:

$ pip install huggingface_hub[cli]
$ huggingface-cli download deepseek-ai/DeepSeek-R1

Downloads to $HOME/.cache/huggingface.

2. Clone SGLang and build the ROCm container (can take up to 30 minutes):

$ git clone https://github.com/sgl-project/sglang.git
$ cd sglang/docker
$ docker build --build-arg SGL_BRANCH=v0.4.2 -t sglang:v0.4.2-rocm620 -f Dockerfile.rocm .

If you hit error: RPC failed; curl 56 GnuTLS recv error during the build, add these lines to Dockerfile.rocm before the repo-cloning steps:

RUN git config --global http.postBuffer 1048576000
RUN git config --global https.postBuffer 1048576000

Connection timeouts during build are usually transient — just re-run; Docker caches completed layers.

3. Run the inference server:

$ docker run -d --device=/dev/kfd --device=/dev/dri --ipc=host \
    --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
    -v $HOME/dockerx:/dockerx -v $HOME/.cache/huggingface:/root/.cache/huggingface \
    --shm-size 16G -p 30000:30000 sglang:v0.4.2-rocm620 \
    python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-R1 --tp 8 --trust-remote-code --host 0.0.0.0 --port 30000

Runs detached with GPU device access, mounted caches, 16GB shared memory, and tensor parallelism across 8 GPUs (--tp 8), serving on port 30000.

4. Test inference:

$ curl http://localhost:30000/v1/chat/completions \
     -H "Content-Type: application/json" \
     -d "{\"model\": \"deepseek-ai/DeepSeek-R1\", \"messages\": [{\"role\": \"user\", \"content\": \"Walk through the reasoning steps for solving a system of two linear equations.\"}], \"temperature\": 0.7}"

5. Optional — expose the port externally:

$ sudo ufw allow 30000

Next Steps

DeepSeek R1 is serving reasoning-focused inference through SGLang's OpenAI-compatible API on port 30000. From here:

• Front the server with a reverse proxy and TLS if exposing it beyond localhost
• Tune --tp to match your available GPU count
• Compare R1's chain-of-thought output against a non-reasoning model on the same prompts to see the difference in approach

For the full guide, visit the original article on Vultr Docs.