.svg)
.svg)
.svg)
.svg)
.svg)
.svg)
.svg)
Ollama is the quickest way to run open models like Llama 3.2, Qwen3 and DeepSeek-R1 on your own hardware. This guide installs it on Ubuntu 24.04, gets a model answering prompts and covers the parts most guides skip: where models land on disk, how to expose the API safely and how to upgrade or remove everything later. The commands are identical on Ubuntu 22.04 and Debian 12 or 13, so follow along on any of them.
Prerequisites
You need a machine with sudo access and curl installed. That's the whole list. Ollama runs fine on CPU alone, so a GPU is optional; it picks up NVIDIA and AMD cards automatically when drivers are present. If you're unsure how much RAM a given model needs, the short version is 8GB for 3B models and 16GB for 7B to 8B, and there's a full breakdown in our Ollama hardware requirements guide.
On a fresh server, install curl first:
sudo apt update
sudo apt install -y curlStep 1: Install Ollama with the official script
There are two install routes on Linux: the official script and a manual tarball. I use the script on every normal server, and I'd tell you to do the same. It detects your architecture, creates a dedicated ollama system user, writes the systemd unit and enables the service, which is four things you'd otherwise do by hand. The manual route exists for a reason (more on that below), but for a standard Ubuntu box the script is the right default.
curl -fsSL https://ollama.com/install.sh | shThe script prints what it's doing as it goes. On a machine with an NVIDIA card but no drivers it will tell you so; install the drivers and the service picks up the GPU on restart without reinstalling anything. Piping curl into sh makes some people twitch, and fair enough. You can download the script first and read it, or use the tarball method further down. The full official steps live in the Ollama Linux documentation if you want to compare notes.
Step 2: Verify the install and the service
Check the binary and the service in one pass:
ollama --version
systemctl status ollamaThe first command prints the installed version. As of the v0.32 line (August 2026) you'll see something in that series. The second should show active (running). The service starts on boot from now on, listening on 127.0.0.1:11434. Quick API sanity check:
curl http://127.0.0.1:11434That returns the string Ollama is running. If it doesn't, jump to the troubleshooting section at the end.
Step 3: Pull and run your first model
Start small. llama3.2:3b is a 2.0GB download, answers general questions well and runs on CPU at a usable speed:
ollama run llama3.2:3bThe first run downloads the model, then drops you into an interactive prompt. Ask it something, then type /bye to exit. The model stays loaded in memory for five minutes after the last request, so a second prompt answers much faster than the first.
Browse the Ollama model library for alternatives; qwen3 has strong small variants and gemma3 handles images. Each model page lists tags with exact download sizes, which matters more than you'd think once you've filled a 40GB disk with four models. Our roundup of the best Ollama models has opinions on what's worth pulling in 2026, and the Ollama commands cheat sheet covers ollama list, ollama ps, ollama rm and the rest.
Where Ollama stores models on disk
With the service install, models live under /usr/share/ollama/.ollama/models. Check what they're costing you:
sudo du -sh /usr/share/ollama/.ollama/modelsPlan disk space before you get enthusiastic. A 3B model takes about 2GB, a 7B around 4 to 5GB (deepseek-r1:7b is 4.7GB, for example) and larger models climb fast from there. I keep at least 30GB free on any box that runs Ollama seriously.
To store models somewhere else, a mounted volume for instance, set OLLAMA_MODELS in the service environment (the systemd override method in the API section below works for this too). The ollama user needs read and write access to the new path, and forgetting that chown is the number one reason the variable "doesn't work".
Manual install from the tarball
Skip this section unless you need it. The tarball route makes sense in two cases: air-gapped servers where piping a script from the internet isn't an option, and environments where you pin a specific version and promote it through staging. Download and extract:
curl -fsSL https://ollama.com/download/ollama-linux-amd64.tar.zst | sudo tar x -C /usrThere's an ollama-linux-arm64.tar.zst build for ARM servers. If tar complains about the compression, install the zstd package and retry. At this point ollama serve runs in the foreground, which is fine for a quick test and useless for a server, so set up the service the script would have created. First the dedicated user:
sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama
sudo usermod -a -G ollama $(whoami)Then create /etc/systemd/system/ollama.service with this content:
[Unit]
Description=Ollama Service
After=network-online.target
[Service]
ExecStart=/usr/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=$PATH"
[Install]
WantedBy=multi-user.targetRestart=always with a three second delay means a crashed daemon comes back on its own, which has saved me a late-night SSH session more than once. Load and enable it:
sudo systemctl daemon-reload
sudo systemctl enable ollama
sudo systemctl start ollamaExpose the API on a LAN or VPS
By default Ollama only answers on localhost. That's the correct default. To reach it from other machines, from a laptop pointed at your VPS or another server on the same private network, override the bind address through systemd:
sudo systemctl edit ollama.serviceAdd this under the [Service] section in the editor that opens:
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"Save, then reload and restart:
sudo systemctl daemon-reload
sudo systemctl restart ollamaA blunt warning before you do this on a public server: Ollama ships with no authentication. Binding to 0.0.0.0 on a VPS with an open firewall hands your GPU or CPU to anyone who scans port 11434, and people do scan for it. Put a firewall rule, a reverse proxy or a VPN in front before exposing anything. The full secure setup with all four options is in our guide to hosting Ollama on a VPS. If you'd rather skip the whole install, the LumaDock Ollama VPS template deploys Ollama with Open WebUI already wired together in one click.
Upgrade Ollama
Script installs upgrade by running the same script again:
curl -fsSL https://ollama.com/install.sh | shYour models, the service unit and any systemd overrides survive the upgrade untouched. Manual installs need one extra step first, removing the old libraries so stale files don't linger:
sudo rm -rf /usr/lib/ollama
curl -fsSL https://ollama.com/download/ollama-linux-amd64.tar.zst | sudo tar x -C /usr
sudo systemctl restart ollamaNew releases land often, sometimes weekly, and they regularly bring day-one support for new models. Release notes are on the Ollama GitHub repository.
Uninstall Ollama completely
Removal is manual either way. Stop the service, remove the unit, then delete the binary, libraries, models and the service user:
sudo systemctl stop ollama
sudo systemctl disable ollama
sudo rm /etc/systemd/system/ollama.service
sudo rm $(which ollama)
sudo rm -rf /usr/lib/ollama
sudo userdel ollama
sudo groupdel ollama
sudo rm -r /usr/share/ollamaThat last line deletes every downloaded model, so copy anything you want to keep first. If you set a custom OLLAMA_MODELS path, clean that directory too.
Troubleshooting the install
Fix ollama: command not found
Right after installing, your shell's command cache is sometimes stale. Run hash -r or open a new terminal. If the command is still missing, confirm the binary exists with ls -l /usr/bin/ollama and check /usr/local/bin as well; a manual tarball extracted to the wrong prefix is the usual cause.
Service fails to start
Read the logs before guessing:
journalctl -e -u ollamaAdd -f to follow live. The two failures I see most: port 11434 already taken by an older ollama serve left running in a terminal (kill it, restart the service), and a custom OLLAMA_MODELS directory the ollama user can't write to (fix with sudo chown -R ollama:ollama /path/to/models). For deeper digging, set Environment="OLLAMA_DEBUG=1" in a systemd override and restart; the logs get much chattier.
Model runs painfully slowly
On CPU, check that the model fits in RAM with free -h while it's loaded. A model that spills into swap crawls. Drop to a smaller quantization or a smaller parameter count, or move to GPU. If you'd rather run everything in containers instead of on the host, the same setup works there too; see our Ollama Docker guide for the CPU and GPU container routes.
