How to run an LLM locally on Windows, Mac or Linux

This guide runs a language model on your own computer with Ollama, a free, open-source tool for Windows, macOS and Linux. With a local model, your prompts are processed on your machine and there is no subscription; Ollama's cloud models are optional and can be switched off. This guide uses the commands from Ollama's own documentation and README, checked on October 1, 2026.

Before you start: will it fit?

Two things limit what you can run:

• Memory. The model has to fit in your GPU's memory (VRAM) or, on Apple Silicon Macs, in the shared system memory. An 8-billion-parameter model needs roughly 6 to 8 GB; our VRAM calculator gives the figure for any model, quantization and context length.
• Disk space. On Windows, Ollama needs at least 4 GB for the program itself, and each model adds its download size on top: from 523 MB for the smallest Qwen3 model to 142 GB for the largest.

You can run models without a GPU, on the processor alone, but they will be much slower. On Intel Macs, Ollama runs on the CPU only.

Step 1: install Ollama

Windows (Windows 10 22H2 or newer). Open PowerShell and run:

irm https://ollama.com/install.ps1 | iex

Or download OllamaSetup.exe from ollama.com. It installs in your user folder without administrator rights, runs in the background and adds the ollama command to your terminal. If you have an NVIDIA card, Ollama needs driver 551.61 or newer.

macOS (Sonoma 14 or newer). In Terminal:

curl -fsSL https://ollama.com/install.sh | sh

Or download Ollama.dmg and drag the app to Applications. On first launch it offers to link the ollama command into /usr/local/bin.

Linux. The same script installs it:

curl -fsSL https://ollama.com/install.sh | sh

For NVIDIA GPUs you also need the CUDA drivers (nvidia-smi should list your card); for AMD, ROCm 7. Ollama's Linux guide also explains how to run it as a systemd service so it starts with the computer.

Check that it works:

ollama -v

Step 2: choose a model

Ollama's library lists each model with its download size. A few starting points, from smallest to largest:

The download size is close to the memory the weights need; the context adds more on top. If you are between two sizes, start with the smaller model.

Step 3: chat with it

ollama run qwen3:8b

You get a prompt where you can type questions. To paste several lines at once, wrap them in triple quotes ("""). Other commands you will use:

Step 4: check that it runs on the GPU

While a model is loaded, run ollama ps. The PROCESSOR column says "100% GPU" when the whole model is in video memory. If it shows a split between CPU and GPU, part of the model is running on the processor and answers will be slower: try a smaller model or a smaller quantization.

Step 5: give it more context (optional)

By default, Ollama uses a 4K context when your GPU has less than 24 GiB of VRAM. That is fine for short questions, but agents, coding tools and long documents need more; Ollama recommends at least 64,000 tokens for those tasks. You can raise it with the slider in the Ollama app's settings, or when starting the server:

OLLAMA_CONTEXT_LENGTH=64000 ollama serve

A longer context needs more memory. Check ollama ps again afterwards.

Step 6: use it from other apps

Ollama runs an API on your computer at http://localhost:11434. A minimal request:

curl http://localhost:11434/api/chat -d '{
  "model": "qwen3:8b",
  "messages": [{ "role": "user", "content": "Why is the sky blue?" }],
  "stream": false
}'

Apps built for OpenAI's API can use Ollama by setting the base URL to http://localhost:11434/v1, and Anthropic-style clients can use http://localhost:11434. Ollama supports a subset of each API, so check its compatibility pages if a feature is missing. The command line can also connect coding agents to Ollama, for example ollama launch claude for Claude Code.

Where the models are stored, and how to move them

• Windows: %HOMEPATH%\.ollama. To store models on another drive, create a user environment variable called OLLAMA_MODELS with the new folder, then quit and reopen Ollama.
• macOS: ~/.ollama.

Troubleshooting

• Squares instead of progress bars on Windows 10: change your terminal font; Ollama uses Unicode characters that some older fonts lack.
• AMD Radeon RX 6000 cards on Windows: some do not expose ROCm 7 with current drivers. Ollama enables Vulkan by default as the fallback.
• Logs: on Windows, %LOCALAPPDATA%\Ollama (server.log); on macOS, ~/.ollama/logs; on Linux with the service, journalctl -e -u ollama.

Prefer a graphical app?

LM Studio does the same job with a desktop interface for browsing and chatting with models. The license is different, which matters at work: see Ollama vs LM Studio.

What we checked

Every command and requirement on this page comes from Ollama's documentation, README and CLI reference, and the download sizes from Ollama's library pages, all checked on October 1, 2026. We have not timed downloads or measured speeds on specific hardware.