.svg)
.svg)
.svg)
.svg)
.svg)
.svg)
.svg)
Last spring I helped someone debug a VPS bill that made no sense. CPU pinned at 100% for nine days straight on a box that "just ran a little AI experiment". The experiment was Ollama bound to 0.0.0.0 with no firewall, and half the internet had found the free LLM endpoint before he did. Scanners index open 11434 ports the way they index open databases, and there are thousands listed on Shodan at any given moment. So this guide does both halves of the job: getting Ollama reachable from your apps and machines, and locking it down so the only people prompting your models are the ones you invited.
Why host Ollama on a server instead of your laptop
A laptop install is great until you close the lid. A server gives you an always-on API endpoint for the things that need one: n8n flows that call a model at 3 AM, agents that run on schedules, a chatbot behind your website. It also gives a team one shared endpoint and one shared model cache instead of five laptops each holding 40GB of models. And the privacy argument is the quiet one that convinces most people: prompts and documents go to hardware you control, in a jurisdiction you chose, with logs you can read. For anyone processing client data under GDPR, that beats shipping context to a third-party API in a way that's easy to explain to an auditor.
Size the VPS honestly
Some plain numbers, because this is where budgets go wrong. A CPU-only VPS runs quantized models in the 3B to 8B range at conversational speed, and that's the realistic ceiling. An 8GB RAM plan handles 3B models with room for the OS; 16GB is the comfortable floor for 7B and 8B models like llama3.1 8B or deepseek-r1:8b (a 5.2GB download that wants roughly that much memory when loaded, plus headroom for context). Cores matter too, since CPU inference scales with them; I wouldn't run 7B on fewer than 4 dedicated-feeling vCPUs and 8 is noticeably better.
Beyond 8B, or when you need fast responses for several users at once, CPU stops being a sensible place to save money and you want a GPU with enough VRAM to hold the model. Full-passthrough cards on a GPU VPS, from a 16GB T4 up to the RTX PRO 6000 Blackwell with 96GB, cover everything from 7B at high speed to 70B-class models. The full sizing tables, model by model and quant by quant, are in our Ollama hardware requirements guide. Disk is the boring one people forget: models are 2 to 5GB each in the small range and you will accumulate them, so start with 80GB or more of NVMe.
Install Ollama on the server
Two lines on Ubuntu 24.04 or Debian:
curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.2:3bThe script sets up the systemd service and a dedicated user; the pull gives you something to test with. Verification steps, the manual tarball route and disk layout are all in our Ollama Ubuntu install guide, so I won't repeat them here.
Expose the API with OLLAMA_HOST
Fresh installs answer only on 127.0.0.1:11434. To accept connections from outside, override the bind address in the systemd service:
sudo systemctl edit ollama.serviceAdd this in the editor that opens, above the comment block:
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"Then apply it:
sudo systemctl daemon-reload
sudo systemctl restart ollamaConfirm the new binding with ss -tlnp | grep 11434; the local address column should show 0.0.0.0:11434 or *:11434. Now test from a different machine:
curl http://YOUR_SERVER_IP:11434/api/tagsYou should get JSON listing your pulled models. If that curl succeeds from your laptop, understand what it means: it succeeds from everywhere. Which brings us to the uncomfortable part.
The security problem with an open port 11434
Ollama has no built-in authentication. None. There are no API keys or accounts, no rate limiting and nothing recording who asked what. Anyone who can reach the port can run generations on your hardware, list your models, pull new ones onto your disk and delete what's there. An open 11434 on a public IP is a free compute donation, and automated scanners find fresh ones within hours. The env vars like OLLAMA_ORIGINS control browser CORS, and that's browser etiquette rather than security; curl ignores it completely. So pick one of the three patterns below before you point anything at your server. All three are compatible with each other, and the firewall one takes two minutes.
Option 1: Firewall allowlist with ufw
The right choice when a known, fixed set of machines calls the API, like one app server or an office IP. Allow SSH first (skipping this line locks you out, and yes, people do it), then allow 11434 only from the callers you trust:
sudo ufw allow OpenSSH
sudo ufw allow from 203.0.113.10 to any port 11434 proto tcp
sudo ufw default deny incoming
sudo ufw enableReplace 203.0.113.10 with your app server's IP; repeat the line per caller. Everything else hitting 11434 gets dropped silently. Verify from an allowed machine (curl works) and a disallowed one (curl times out). The limits are practical rather than cryptographic: home IPs change, and there's no encryption on the wire, so prompts cross the internet as plaintext. Fine between two servers in the same datacenter, weak for a team on laptops. One Docker-specific warning: if you later run Ollama in a container, published ports bypass ufw entirely, so bind the container to 127.0.0.1 and proxy it instead.
Option 2: Nginx reverse proxy with TLS and basic auth
The pattern I use for anything multi-user or reachable from changing locations. Ollama stays on localhost (revert the OLLAMA_HOST override, or never apply it), and Nginx terminates HTTPS on 443, checks a password and forwards to 11434. You need a domain pointed at the server. Install the pieces and create a credential:
sudo apt install -y nginx apache2-utils certbot python3-certbot-nginx
sudo htpasswd -c /etc/nginx/.htpasswd apiuserCreate /etc/nginx/sites-available/ollama:
server {
listen 80;
server_name ollama.example.com;
location / {
auth_basic "Ollama API";
auth_basic_user_file /etc/nginx/.htpasswd;
proxy_pass http://127.0.0.1:11434;
proxy_http_version 1.1;
proxy_set_header Host localhost:11434;
proxy_read_timeout 600s;
proxy_buffering off;
}
}Four directives here are load-bearing. proxy_http_version 1.1 keeps connections alive properly for the API. proxy_read_timeout 600s stops Nginx from killing long generations at its 60 second default, which otherwise surfaces as mysterious mid-answer 504s on big models. proxy_buffering off makes streaming work, so tokens reach the client as they're produced instead of arriving in one lump at the end. And the Host header is set to a local value because Ollama inspects it as protection against DNS rebinding; if you ever see 403s through a proxy that works with curl directly, this header is the first thing to check. Enable the site and fetch a certificate:
sudo ln -s /etc/nginx/sites-available/ollama /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
sudo certbot --nginx -d ollama.example.comCertbot rewrites the block for HTTPS and installs a renewal timer. Test the finished thing:
curl -u apiuser:yourpassword https://ollama.example.com/api/tagsWrong credentials get a 401, everything is encrypted in transit and your clients just need the URL plus basic auth. We use the same Nginx shape for other self-hosted APIs; the n8n Nginx reverse proxy guide and the Hermes agent HTTPS setup are the same idea with different upstreams, worth a look if you're proxying several services on one box.
Option 3: Keep it private with WireGuard or Tailscale
The strongest pattern, and the one I'd pick for a personal server or a small team: never expose the API to the internet at all. An overlay network like WireGuard or Tailscale gives your devices and the VPS private addresses on an encrypted mesh. Then bind Ollama to the tunnel interface instead of all interfaces. With a WireGuard server address of 10.8.0.1, the systemd override becomes:
[Service]
Environment="OLLAMA_HOST=10.8.0.1:11434"Reload, restart and the API exists only inside the tunnel; a scan of the public IP shows nothing on 11434. Clients use http://10.8.0.1:11434 (or the Tailscale IP) exactly as they'd use a local instance. There are no certificates or passwords to manage, and nothing exposed on the public IP beyond WireGuard's own UDP port. The tradeoff is that every caller needs to be on the mesh, which is trivial for your own devices and an extra onboarding step for teammates or third-party webhook services. In practice I combine options: mesh for humans, a firewall allowlist for the one app server that can't join it.
Tune Ollama for multiple users
Once several people or services share the endpoint, three environment variables decide how it behaves under load, all set via the same systemd override and documented in the Ollama FAQ.
OLLAMA_NUM_PARALLEL (default 1) sets how many requests one loaded model serves at once. Raising it to 4 lets four chats run concurrently instead of queueing, at the cost of memory, since the KV cache multiplies with parallel slots. OLLAMA_MAX_LOADED_MODELS caps how many different models sit in memory together (default 3 on CPU, 3 per GPU otherwise). Each loaded 7B model holds its 4 to 5GB even when nobody is talking to it, so on a 16GB box I set this to 1 or 2 deliberately rather than letting three models fight the OS for RAM. OLLAMA_KEEP_ALIVE (default 5 minutes) decides how long an idle model stays resident: raise it for snappy responses all day on a dedicated inference box, lower it or leave the default when Ollama shares the server. Requests beyond the queue limit (OLLAMA_MAX_QUEUE, default 512) get a 503, which well-behaved clients should retry with backoff.
For human users, put a frontend with real accounts in front of the API rather than sharing basic auth credentials around; our Open WebUI setup guide covers that, and the UI's login replaces the need to expose 11434 to browsers at all. For programmatic access, both the native endpoints and the OpenAI-compatible ones are walked through in the Ollama API guide, including how existing OpenAI SDK code points at your server with a one-line base URL change.
Operations checklist
The short list I run on my own Ollama boxes, monthly or after anything weird:
- Logs:
journalctl -e -u ollamafor recent entries,-fto watch live during a problem. - Disk:
sudo du -sh /usr/share/ollama/.ollama/models, thenollama listandollama rmfor models nobody has touched in months. Model bloat is the most common way these servers fill their disks. - Memory:
free -hwhile models are loaded. Swap usage above zero during inference means you're oversubscribed; unload a model or resize. - Updates: rerun the install script for Ollama,
apt upgradefor Nginx and the OS, and check certbot renewed withsudo certbot renew --dry-run. - Exposure:
ss -tlnpto confirm nothing rebound 11434 to a public interface after a config change, and an external port scan of your IP once in a while to confirm the world sees what you think it sees.
That last check is the one that catches the drift between what you configured and what's running. A systemd override lost in a migration, a helpful teammate who "fixed" the binding, a Docker port published wide: they all show up in one nmap from your laptop, and finding it yourself beats the nine-day CPU bill every time.
