How to run an LLM locally in 2026
Running a large language model on your own computer stopped being a hobbyist stunt a while ago. In 2026, a mid-range gaming PC or a recent Mac runs models that handle most everyday AI work: coding help, summarizing, drafting, answering questions over your own documents, and powering automation. You pay nothing per request, nothing per month, and your data never leaves the machine.
This guide takes you from nothing to a working local model in under an hour. No cloud account, no API key.
Step 1: Check what your hardware can run
The single number that matters most is memory. If you have a dedicated GPU, it is the video memory (VRAM). If not, it is your system RAM. Match your number to a model class:
| Your memory | Model class | Example |
|---|---|---|
| 8 GB RAM, no GPU | 3B models | llama3.2:3b |
| 16 GB RAM, no GPU | 8B models | llama3.1:8b |
| 12 GB VRAM or 32 GB RAM | 14B models | qwen2.5:14b |
| 16-24 GB VRAM or 48 GB unified (Mac) | 32B models | qwen2.5:32b |
Two rules of thumb: a quantized model takes roughly 0.6 GB per billion parameters, and VRAM beats RAM. A 12 GB GPU will outrun 64 GB of CPU-only RAM. For the full breakdown, see our hardware requirements guide.
Step 2: Install Ollama
Ollama is the easiest way to run models locally in 2026. It handles downloading, quantization formats, and GPU acceleration for you, and it exposes a simple local API other tools can use.
Mac: download the installer from ollama.com, or brew install ollama.
Windows: download the installer from ollama.com and run it.
Linux:
curl -fsSL https://ollama.com/install.sh | shRead the script before you pipe it to a shell. That habit will serve you well through everything else on this site.
Step 3: Pull a model and talk to it
Pick the model from the table that fits your memory, then:
ollama pull qwen2.5:14b
ollama run qwen2.5:14bThe pull is a one-time download of several gigabytes. When the prompt appears, everything you type is being answered by a model running entirely on your machine. Unplug the network cable if you want to prove it to yourself. People usually do this exactly once, grinning.
Step 4: Decide what it is for
A chat prompt is a demo. The value shows up when the model does work. Some starting points, roughly in order of effort:
- A private chat assistant that never logs your questions anywhere.
- Coding help in your editor. Most editors can point at Ollama's local API.
- Search over your own notes and documents, with answers that cite your files.
- Agents: small programs where the model watches for a trigger and takes an action, like summarizing every file dropped into a folder. See our guide to running AI agents locally before you give any agent real access.
What local models can and cannot do
Honesty matters here. The biggest cloud models are still smarter on the hardest problems. A 14B model on your desk will not match them on graduate-level reasoning or very long, intricate tasks. What it will do is handle the large majority of day-to-day work at zero marginal cost, with total privacy, and with no rate limits. Most experienced local users end up with a hybrid setup: local by default, cloud for the rare problem that needs it. That combination is cheaper than cloud-only by a wide margin.
Want the fast path? Our free starter kit is a script that detects your hardware, installs Ollama, pulls the right-sized model, and includes a working sandboxed agent plus a printable sizing chart. Get the starter kit. No credit card, no catch.
Common first-hour problems
The model is slow. You are probably running on CPU. Check that your GPU is detected, or drop one model class down.
Out of memory. The model is too big for your hardware. Pull the next class down. A smaller model that runs beats a bigger one that crashes.
The answers are mediocre. Model choice matters more than most settings. As of mid-2026, the Qwen 2.5 family punches above its weight per gigabyte, and new releases land monthly. This changes fast, which is one reason communities beat static guides.