Ollama: How to Run AI Locally on Your Computer for Free

An AI assistant that runs entirely on your own computer. No subscription, no API bill, no internet needed once it is downloaded, and nothing you type ever leaves your machine. That is not a future promise; you can have it working in about ten minutes.

The tool is Ollama. This guide installs it on Windows, Mac and Linux with the actual commands, helps you choose a model that fits your RAM instead of one that crawls, and shows you how to use it from your own code.

What Ollama Actually Does

When you use a cloud AI service, your words travel to somebody else’s server, get processed there, and come back. With Ollama the model file sits on your disk and runs on your own processor and memory.

1Install Ollama

Windows

Go to ollama.com/download, get the Windows installer, run it and follow the steps. When it finishes, Ollama runs quietly in the background and its icon appears in the system tray near the clock.

macOS

Download the macOS file from the same page, open it, and drag Ollama into Applications. Launch it once; the icon appears in the menu bar at the top of the screen. If you prefer Homebrew: brew install ollama.

Linux (and WSL)

The original guide said to “run the installation command” without showing it. Here it is:

curl -fsSL https://ollama.com/install.sh | sh

Check it worked

ollama --version

$ ollama --version
ollama version is 0.12.3

Example output. Any version number means you are ready.

2Choose a Model That Fits Your Computer

This is where most people go wrong. They install the biggest model they have heard of, it crawls at one word per second, and they decide local AI is not viable. The model must fit comfortably in your memory, with room left for everything else you are running.

3Run Your First Model

ollama run llama3.2

The first run downloads the model, which is a few gigabytes, so it takes a while. After that it starts in seconds and never needs the internet again.

$ ollama run llama3.2
pulling manifest
pulling dde5aa3fc5ff: 100% ▕████████████████▏ 2.0 GB
verifying sha256 digest
writing manifest
success

>>> Explain what a Docker volume is, in two sentences.

A Docker volume is a storage area managed by Docker that lives
outside a container's own filesystem. Because it is separate, the
data inside it survives when the container is deleted or rebuilt.

>>> /bye

Example session. Type /bye to leave, or /? to see the in-chat commands.

Models worth knowing

4The Commands You Will Use

$ ollama list
NAME                  ID              SIZE      MODIFIED
llama3.2:latest       a80c4f17acd5    2.0 GB    2 hours ago
qwen2.5-coder:7b      2b0496514337    4.7 GB    yesterday
mistral:latest        f974a74358d6    4.1 GB    3 days ago

$ ollama ps
NAME                  SIZE      PROCESSOR          UNTIL
llama3.2:latest       3.1 GB    100% GPU            4 minutes from now

Example output. In ollama ps, “100% GPU” means it is running on your graphics card, which is much faster. “100% CPU” means it is not.

5Use It From Your Own Code

Ollama quietly runs a local API on port 11434. This is what turns it from a terminal toy into something you build with: no key, no bill, and your data never leaves the machine.

curl http://localhost:11434/api/tags

const res = await fetch('http://localhost:11434/api/chat', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model: 'llama3.2',
    stream: false,
    messages: [
      { role: 'system', content: 'You summarise text in exactly 3 bullets.' },
      { role: 'user',   content: '...paste the text here...' }
    ]
  })
});

if (!res.ok) throw new Error(`Ollama error ${res.status}`);

const data = await res.json();
console.log(data.message.content);

<?php
$payload = [
  'model'  => 'llama3.2',
  'stream' => false,
  'prompt' => 'Write a one-line product description for a blue mug.',
];

$ch = curl_init('http://localhost:11434/api/generate');
curl_setopt_array($ch, [
  CURLOPT_POST           => true,
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_HTTPHEADER     => ['Content-Type: application/json'],
  CURLOPT_POSTFIELDS     => json_encode($payload),
]);

$data = json_decode(curl_exec($ch), true);
echo $data['response'];

6Make Your Own Assistant

A Modelfile wraps an existing model with your own instructions and settings. Create a file named Modelfile, with no extension:

FROM llama3.2

# 0 is focused and repeatable, 1 is creative and loose
PARAMETER temperature 0.3

# how much conversation it can remember at once
PARAMETER num_ctx 4096

SYSTEM """
You are a senior Laravel developer reviewing code for a junior.
Always: point out security problems first, keep answers under
150 words, and show a corrected code snippet.
Never: suggest a package without saying why it is needed.
"""

ollama create laravel-reviewer -f Modelfile
ollama run laravel-reviewer

Now you have a permanent assistant with that personality, usable from the terminal or the API like any other model.

Want a Chat Window Instead of a Terminal?

Ollama is a command-line tool at heart, but several free interfaces sit on top of it and look like a normal chat app.

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui --restart always \
  ghcr.io/open-webui/open-webui:main

# Then open http://localhost:3000

When It Does Not Behave

Being Honest About the Limits

Local models are genuinely useful. They are not the same as the largest cloud models, and pretending otherwise leads to disappointment.

Common Questions

The Full Video Guide

Here is the whole setup on video:

New AI, Linux, Docker and Laravel tutorials every week.

If this was useful, share it with someone paying monthly for something their laptop can do.