I built Infercat: Share your local AI with friends over an encrypted p2p tunnel. Connect with an invite code from anywhere.

Hello LocalLLaMA Fam! I built this project because I, like many of us here, enjoy local AI and have a nice setup, but still find myself reaching for Cloud-based AI due to their convenience. My models run on my MacBook and Linux server, but the experience is disjointed - I don't bother using my own local AI when I am on my phone - I just use ChatGPT. But what if we could turn our local AI hardware into mini clouds? and make local AI so easy to use that we can recommend it to our family and friends? This is the goal of Infercat (inference + cat, like the unix tool that streams the contents of a file). It's a gateway + tunnel that sits in front of your inference engine and generates each of your friends (or devices) an invite code. Just paste the code into a web page or cli and chat with your model. No account, no system-level VPN, nothing to install when chatting through the browser. Infercat web demo: end-to-end encrypted private chat with local AI model How Infercat works Most of the networking heavy-lifting is done using tailcat (S/O to the awesome folks at tailscale for open sourcing this!), which creates p2p tunnels between any two devices with internet connection. (in this case, your host machine and your friend's browser). A server is needed to facilitate the initial packet exchange but otherwise, the tunnel is an e2e encrypted direct connection between the host and the client. If the client is a web browser, due to technical limitations that will be resolved soon, the server currently acts as the relay after the initial handshake. Even in relay mode, the server only sees the encrypted text, making the tunnel fully private. The invite code is where the magic happens - it combines the tunnel address (your host's public key and where to find it, same for every friend, not a secret) and a gateway key (your friend's API key, like OPENAI_API_KEY ; you keep only a hash). You can manage the gateway keys like a real AI provider: set limits on each key like tokens per day or a model allowlist. Pause, rotate or revoke one friend without touching the others. You host machine records stats, never the conversation transcripts. . Exactly three routes are reachable through the tunnel: /v1/models , /v1/chat/completions and /v1/embeddings . We currently support popular inference engines: llama.cpp, vLLM, Ollama and LM Studio. Discovery is automatic or configurable. The gateway is OpenAI-compatible, so infercat connect turns an invite into a local 127.0.0.1:11435/v1 for Open WebUI, Cursor, Claude Code or plain curl. Get started by running: curl -fsSL infercat.ai/install.sh | sh # or: brew install infercat/tap/infercat infercat serve # finds llama.cpp, vLLM, Ollama or LM Studio infercat keys add alice # prints alice's invite once, plus a QR Roadmap Currently we validated the CLI as an OpenAI API tunnel and the web application as a text-only chat interface. We have a few ideas on where to go next and would love to hear your feedback and feature requests! One lane is to make hosts more powerful "mini-clouds" a monitoring / management dashboard support for multiple models new model types such as: ocr, asr, image generation, video generation server-side tools: search, exec local sandboxes ... any other feature that you can find in a big "AI Cloud" A second lane is to make the AI service easier to consume AI gateway server on the public internet (instead of connecting to a tunnel, just connect to gateway.infercat.ai/h/[hostname]/v1/chat/completions , with all requests forwarded to your host) Web App improvements (Image/File upload, Agent harness...) New Mobile App Clients programmatic libraries (TS, GO, etc.) ... any other feature on the client side. Would you use this? Does the invite code model make sense? I want to solve connectivity / distribution for local AI and would love to hear any ideas and use cases. All feedback is welcome, including "nobody wants this" :) Try it out! Infercat is open-source under the MIT License. Code: github.com/infercat/infercat Site and demo: infercat.ai