.svg)
.svg)
.svg)
.svg)
.svg)
.svg)
.svg)
Ollama is an MIT-licensed runtime for running large language models on your own hardware. You install it, type ollama run llama3.2 and you're chatting with a model that lives on your disk, answers without an internet connection and never sends a prompt to anyone's cloud. It has become the default way to run local LLMs for a plain reason: it removes every excuse. There's no Python environment to assemble and no model files to hunt down by hand. One installer, one command, a working model.
The pitch in practical terms: your data stays on your machine, which matters for anything under NDA, GDPR or plain personal privacy. Your costs are hardware you already own or rent, with no per-token meter running. And your applications talk to it through an OpenAI-compatible API, so code written for cloud models points at localhost with a one-line change. Those properties explain most of the project's enormous popularity, and the rest is explained by it simply being pleasant to use.
What do people run on it? More than chat. Coding assistants in editors that autocomplete against a local model. Summarization pipelines chewing through contracts and support tickets that legally can't leave the building. Agents and automation tools calling a model hundreds of times a night at a marginal cost of zero. Home lab dashboards, Discord bots, RAG setups over private wikis and a long tail of scripts where a mediocre free model beats an excellent metered one. The pattern across all of them: high volume, privacy constraints or both.
This guide is the map of our whole Ollama series. Read the first two sections for the mental model, then jump wherever your situation points.
How Ollama works
Four pieces, once you see them, make everything else predictable.
A registry. Models come from the Ollama library, pulled by name and tag the way Docker pulls images: ollama pull qwen3:8b fetches the 8B variant, layers and all, with resume on interruption. Tags encode size and quantization, so you pick the variant that fits your memory rather than the one true file.
Quantized GGUF models. The library serves models compressed to 4-bit and similar precisions in the GGUF format. Quantization is the technology that makes local AI practical at all: it shrinks an 8B model from 16 GB to roughly 5, with a quality cost small enough that most people never notice it in daily use.
The llama.cpp engine. Inference itself runs on llama.cpp, the C/C++ engine the whole local AI world builds on. Ollama bundles it, keeps it updated and handles GPU offload automatically, splitting a model between VRAM and RAM when it doesn't fully fit.
A client-server design. The ollama command you type is a thin client. The real work happens in a server process listening on 127.0.0.1:11434, which loads models on demand, keeps them warm for five minutes after use and exposes the API every tool integrates against. This is why Ollama feels like infrastructure rather than an app, and why everything from editors to chat UIs plugs into it.
Installing Ollama on Ubuntu or in Docker
On Linux, one command installs the binary and registers a systemd service, and our guide to installing Ollama on Ubuntu covers the install, the service management and the upgrade path, which is where first-timers usually stumble. If your infrastructure is containerized, the Ollama Docker guide walks through the official image on CPU, NVIDIA and AMD, including the volume setup that stops you re-downloading 20 GB of models every time you recreate a container. macOS and Windows get native installers from the same download page, with the same commands afterwards.
Pick one install method per machine and stay with it. Mixing the native service and a Docker container on the same box leads to two servers fighting over port 11434, a classic self-inflicted wound.
Adding a web UI with Open WebUI
The terminal is fine for testing and wrong for daily conversation. The standard pairing is Open WebUI, a self-hosted chat interface with conversation history, model switching, document uploads and multi-user accounts, and the Open WebUI and Ollama setup guide takes you from nothing to a working ChatGPT-style interface in about fifteen minutes. It runs as a Docker container next to Ollama, and connecting the two is the one step where people trip (container networking, mostly), so follow the guide's exact flags. If you'd rather skip the assembly entirely, the Ollama VPS template deploys Ollama together with Open WebUI in one click on NVMe-backed hardware, which is the fastest route from zero to a private chat instance you can reach from any browser.
Choosing hardware and models
Memory is the whole game. A model needs to fit in VRAM for speed or RAM for patience, and everything else about hardware selection is detail. The Ollama hardware requirements guide maps model sizes to real RAM and VRAM numbers so you can size a machine before buying or renting it, rather than after the out of memory error.
On the model side, the honest answer to "which model should I run" changes every few months, which is why we keep a living list of the best Ollama models by task, covering coding, chat, summarization and small-hardware picks. Two families earned dedicated guides. The DeepSeek R1 guide covers the reasoning model whose distilled variants run on modest hardware, from a 1.5B toy up to serious 32B and 70B tags. And the gpt-oss guide covers OpenAI's open-weight models, where the 20B fits in 16 GB of RAM and the 120B wants a single 80 GB class GPU, a nice illustration of how wide the local hardware spectrum now runs.
Everyday commands, context and the API
A dozen commands cover normal life: pull, run, list, ps, stop, rm and friends. The Ollama commands cheat sheet collects them with the flags worth memorizing, and it's the page to keep open during your first week.
Two defaults deserve early attention because they bite quietly. First, context: models run with a 4096-token window by default no matter what the model supports, and long documents get silently truncated until you raise num_ctx. The context window guide explains the setting and its very real memory cost. Second, the API: beyond the chat endpoints, it handles model management, embeddings and streaming, and it speaks both its native dialect and the OpenAI-compatible one. The Ollama API guide covers both, with working curl and Python examples, and it's the doorway to using local models from your own code, which is where Ollama stops being a toy and starts being a tool.
Running Ollama on a server
Sooner or later you'll want the models on a machine that isn't your laptop: always on, reachable from everywhere, maybe shared with a team. Ollama's client-server design makes this natural, but the defaults assume localhost, and exposing port 11434 raw to the internet hands an unauthenticated API to whoever finds it. The guide to hosting Ollama on a VPS covers the whole move: binding with OLLAMA_HOST, putting a reverse proxy with authentication in front, firewalling and the sizing questions specific to rented servers. Do that one properly. It's the difference between a private AI server and a public compute donation.
Troubleshooting common Ollama errors
Things do break occasionally, and the failure modes are wonderfully repetitive: a 500 from a crashed model runner, a pull stuck at 92%, a proxy variable poisoning localhost, Open WebUI showing a connection error while Ollama sits there running fine. The Ollama troubleshooting guide catalogs the verbatim error strings with ranked causes and exact fix commands, and it starts with the four diagnostic commands that solve half of everything. Bookmark it before you need it.
How Ollama compares to the alternatives
Three comparisons come up constantly, and they resolve to three different questions. Against LM Studio the question is desktop app or background service, with a real Apple Silicon speed angle, and the Ollama vs LM Studio comparison lands on verdicts per person rather than a diplomatic tie. Against llama.cpp the question is how much control you want over an engine Ollama already uses internally, unpacked in Ollama vs llama.cpp. And against vLLM the question is one user or many at once, since the two tools optimize for opposite traffic shapes, covered with concurrency specifics in Ollama vs vLLM. If you're skimming: desktop explorers can go either way, developers and server operators pick Ollama, production apps serving crowds graduate to vLLM.
Ollama limitations to know upfront
A fair guide lists the sharp edges, and Ollama has four worth naming. There's no built-in authentication, so anything beyond localhost needs a reverse proxy or firewall doing the guarding for it. The default context window is 4096 tokens regardless of what the model could handle, a conservative memory choice that surprises everyone the first time a long prompt gets truncated. Concurrency defaults to one request per model at a time with a queue behind it, fine for individuals and wrong for high-traffic serving. And model quality is bounded by your memory: an 8B quant on a laptop is genuinely useful and still noticeably below the frontier cloud models, which is a trade you make knowingly, for privacy and control, rather than discover disappointed.
None of these are flaws so much as scope. The official FAQ documents most of them plainly, which I count in the project's favor.
Ollama versions and upgrades
Ollama releases frequently, and as of August 2026 the v0.32 line is current. The release cadence matters more than the numbers: new model architectures usually require a runtime update, so "my new model throws an error" is answered by upgrading more often than by anything else. On Linux the upgrade is the install script run again, models untouched, thirty seconds of downtime. I'd suggest upgrading when a model you want demands it or a release note names a fix you need, rather than chasing every version on a server that's working. Boring version management is a feature of this project, and one worth preserving in your own habits.
Two environment variables are worth knowing from day one because nearly every customization routes through them or their siblings: OLLAMA_HOST controls where the server listens, and OLLAMA_MODELS moves the model directory, which on a service install lives at /usr/share/ollama/.ollama/models and grows large enough to deserve a deliberate disk decision. The rest of the variable family (keep-alive, parallelism, origins for CORS) follows the same pattern of a systemd override or a Docker -e flag, so once you've set one you can set them all.
Your first model
If you're new: install it today on the machine you already have, pull a small model like llama3.2:3b and talk to it for ten minutes. That first run answers the question no article can, which is how these models feel on your hardware with your prompts. Then follow the path that matches what you felt: a bigger model (hardware guide), a nicer interface (Open WebUI), your own code against the API, or a server so it's always there. The whole series above is sequenced for exactly that walk, and the first command takes less time than reading this paragraph did.
