.svg)
.svg)
.svg)
.svg)
.svg)
.svg)
.svg)
Updated and tested 22 October 2026
Run nvidia-smi on the machine before you commit to a second month. The header tells you the driver version and the CUDA version it supports, the table tells you the card's real name and how much memory it has, and the "Virtualization Mode" line in nvidia-smi -q tells you if you have the whole GPU (Pass-Through) or a slice of one (VGPU). Everything about which of the three rental models fits you follows from those lines, and it takes ten seconds to check. This article is about the three models: a KVM virtual machine with a physical GPU passed into it, a bare-metal server with one or more GPUs in its slots and an hourly instance on a cloud GPU marketplace. They differ in what you can run, what you're responsible for and how the bill behaves.
Commands assume Ubuntu 24.04 LTS on the guest.
The three models
A GPU VPS is a KVM guest on a shared host, exactly like the standard product described in KVM VPS explained, with one addition: the host uses VFIO (the Linux kernel's framework for handing a PCI device to userspace) to give the VM direct control of a physical GPU. The card's memory and command queues map straight into the guest; the hypervisor gets out of the way for GPU traffic. The rest of the VM (CPU, RAM, disk, network) is still virtualised and still shared with neighbours on the host.
A dedicated GPU server is a physical machine with GPUs in its PCIe slots and nobody else on it. You get the cards, the CPU, the memory, the drives and the responsibility for all of them, as described in VPS vs dedicated server, plus the things only a whole chassis makes possible. Those are covered below, and they're the only reason to pick this model for a single card.
A cloud GPU is an instance on a hyperscaler or a GPU marketplace, billed by the hour or minute, that you start for a job and stop when it's done. It might be a passthrough VM underneath, or bare metal, or a container on someone's bare metal; the point is the billing and the catalogue of card types, up to the eight-way H100 and B200 machines that nobody rents monthly for a hobby.
Passthrough vs vGPU and MIG
NVIDIA's own definition, from the vGPU software user guide, is that in pass-through mode "an entire physical GPU is directly assigned to one VM, bypassing the NVIDIA Virtual GPU Manager" and "the GPU is accessed exclusively by the NVIDIA driver running in the VM to which it is assigned." One card, one VM, the normal datacenter driver inside the guest, the full CUDA toolkit, every video encode and decode engine and all of the VRAM. From inside the VM it is indistinguishable from the card in a physical workstation, and that's the property the whole GPU VPS product rests on.
vGPU is the alternative. The host runs NVIDIA's Virtual GPU Manager, the card is carved into profiles with a fixed slice of frame buffer each (a 16 GB card becomes four 4 GB vGPUs, say) and the GPU's compute engines are time-sliced across the VMs that hold those profiles. Each guest sees a virtual GPU with a driver that needs a licence from NVIDIA's licensing server. It's the right technology for a hundred virtual desktops that each want a little 3D, and the wrong one for a single inference server, because your kernels wait for the scheduler whenever a neighbour's kernels are running, and the slice of VRAM is a hard wall.
MIG (Multi-Instance GPU) is a third thing, available on Ampere and later datacenter parts: the card is partitioned in hardware into up to seven instances with their own memory, cache and compute units, so there's no time-slicing between them. The RTX PRO 6000 Blackwell Server Edition, for example, supports "up to four fully isolated instances" per NVIDIA's spec page. MIG-backed vGPUs exist too. Either way, a MIG slice is a fraction of a card sold as a whole one, and the memory figure on the plan is the number to read.
Microsoft frames the same choice for Hyper-V in its guide to planning GPU acceleration in Windows Server: DDA (Discrete Device Assignment) is "dedicated only" with the VM running "on the native driver", and GPU partitioning, new in Windows Server 2025, is the sharing model. Different vendor, same two shapes.
How to tell what you were sold: nvidia-smi -q | grep -i virtualization. Pass-Through is a whole card. VGPU is a slice. On a MIG-enabled card nvidia-smi -L lists the instances under the GPU.
VRAM as the sizing variable
For LLM inference, image generation and most of what people rent GPUs for in 2026, the card's memory decides what runs at all, and compute decides how fast. Sizing starts with VRAM.
The rule I use for a language model: parameter count times bytes per weight, plus room for the KV cache, plus a gigabyte or two of overhead. A 7 billion parameter model is 14 GB at 16-bit, about 7 GB at 8-bit and under 5 GB at a 4-bit quantisation. The KV cache grows with context length and batch size; NVIDIA's inference optimisation guide gives the per-token size as "2 * (num_layers) * (num_heads * dim_head) * precision_in_bytes" and works the Llama 2 7B case out to roughly 2 GB for a full 4,096-token context at 16-bit. So a 4-bit 7B model with a long context fits comfortably on a 16 GB card, a 13B model at 4-bit fits with less room, and a 70B model needs a 4-bit quantisation and something with 40 GB or more, or two cards. The Ollama hardware requirements guide runs through this per model size. When the model doesn't fit, layers spill to the CPU, and the runtime comparison in Ollama vs llama.cpp shows how badly that hurts and what the offload settings do about it. Partial offload is usable for a chat window and useless for a batch job.
NVIDIA's same guide makes the point that model execution is "frequently memory-bandwidth bound", which is why two cards with the same VRAM can differ two-fold in tokens per second: GDDR6 on an L4 (300 GB/s) moves data at a fraction of the rate of GDDR7 on an RTX PRO 6000 Blackwell, and HBM on an H100 is faster again. Memory size decides if the job runs at all; bandwidth decides how fast, and the spec sheet lists the two separately for a reason.
For Stable Diffusion and ComfyUI the same logic holds with smaller numbers: SDXL at 16-bit wants around 8 GB with headroom, Flux-class models want 16 GB or aggressive quantisation, and video models are what push people to 24 GB and up.
PCIe lanes, NVLink and multi-GPU
A single passed-through card in a VM has whatever PCIe link the host gives it, usually x16 on a modern board, and for one card that's plenty. Multi-GPU is where the models diverge. Two GPUs in one VM is possible on a well-built host, but the cards talk to each other over PCIe through the host's root complex, and the provider decides which slots and which CPU socket they hang off.
NVLink, the direct GPU-to-GPU link, is a bare-metal concern. It needs cards that support it (the SXM form-factor datacenter parts, and a few PCIe cards with a bridge), a board designed for it and a provider who will let you have the whole machine. No VPS product I know of offers it, and the PCIe workstation-class cards, including the current RTX PRO line, don't have it at all. If your training run needs tensor parallelism across four or eight cards with fast interconnect, you're renting a bare-metal box or an hourly cloud instance built for exactly that, and the VPS was never in the running.
Driver and CUDA responsibility
With passthrough the driver is yours to install, and this is where a lot of first-week tickets come from. The host has no NVIDIA driver on the card (it holds it with vfio-pci); the guest needs the datacenter driver matching the card, then the CUDA toolkit, then whatever framework sits on top. On Ubuntu 24.04:
sudo apt update
sudo ubuntu-drivers list --gpgpu
sudo ubuntu-drivers install --gpgpu
sudo rebootThen nvidia-smi should print the card. The CUDA version in that header is the highest the driver supports, and a PyTorch or TensorFlow wheel built against a newer CUDA will refuse to see the GPU. Match the framework's CUDA build to the driver, or upgrade the driver first. Container users skip most of this with the NVIDIA Container Toolkit, which lets a Docker container use the host driver while bringing its own CUDA userspace, and it's the setup I'd pick for anything that has to survive a driver update.
On a dedicated server the same applies, plus firmware, plus cooling, plus the fact that the card sits in a chassis you chose. On a cloud GPU the image usually ships with the driver installed and the provider's supported CUDA versions listed, which is one of the things the hourly price is paying for.
Passthrough has one hard limit: a VM with a PCI device assigned can't be live-migrated. If the host needs maintenance, the VM is stopped and started, and a provider's uptime story for a GPU VPS is a reboot story.
When hourly cloud GPU wins
Training. A fine-tune that runs for six hours on an A100 or H100, once, is the textbook hourly job, and paying a month for that card to sit idle the other 29 days is waste. The same goes for a batch of renders, a one-off dataset embedding run, or evaluating three model sizes before picking one.
Anything H100-class or larger. Those cards live in eight-way systems with NVLink and are priced and cooled accordingly; hourly is how most people can touch one at all.
Experiments. When you don't yet know which card size you need, an afternoon on a marketplace across three instance types answers the question for less than a monthly commitment to the wrong one.
