Using ui-mcp-proxy in Llama-cpp

Yesterday I developed a simple MCP server, which allows local LLMs to call tools that I have written in Rust code. This worked well, but I ran into CORS errors—CORS is a security feature on web browsers. I found that llama-cpp has a special feature to avoid these issues.

Llama-cpp ui-mcp-proxy

With ui-mcp-proxy, passed as an argument to llama-server, a proxy server is set up so that the MCP server is on the same server as llama-cpp. This eliminates the CORS errors altogether.

I was able to remove the middleware logic that set up CORS headers—it wasn't a huge win, but anything that simplifies code is a benefit. After all, code is a burden and having the minimal amount of necessary code will make life easier.

According to the Model Context Protocol specification, JSON-encoded messages must be used, so there will always be some complexity due to decoding and encoding messages. That is essential complexity here.