Ollama Explained: What It Is and How to Install It

You want to run an AI model on your own machine, keep your data off someone else's servers, and stop paying per token for every experiment. Ollama is the tool most people reach for first, and the setup is shorter than the docs make it look. This guide covers what Ollama actually is, how to download and install it on macOS, Windows, and Linux, and how to run your first model in under ten minutes.

By the end you will have a local model answering prompts in your terminal, a mental model for which models fit your hardware, and a clear picture of where a raw local model stops being useful on its own.

Key takeaways

• Ollama is a free, open-source runtime that downloads and runs open-weight LLMs (Llama, Mistral, Gemma, Qwen, and others) locally, with no account and no per-token cost.
• Install it from the official site or a package manager, then run ollama run llama3.2 to pull and start a model in one command.
• Everything runs on your machine. Prompts and responses never leave your computer unless you wire Ollama to something that sends them out.
• Model size is gated by RAM and VRAM. A 7B-8B model wants roughly 8GB free; larger models need more.
• Ollama exposes a local API on port 11434, so other apps can call your models the same way they call a cloud API.
• A raw local model has no memory of you or your work. That is a separate problem, and the fix is context, not a bigger model.

---

What is Ollama

Ollama is a lightweight runtime that lets you download and run open-weight language models on your own hardware. Think of it as a package manager plus a server for LLMs. You ask for a model by name, Ollama pulls the weights, quantizes and loads them, and gives you a chat prompt in your terminal and a local HTTP API. It is free and open source, released under the MIT license.

The word "open-weight" matters. These are models whose trained weights are published for anyone to download, such as Meta's Llama family, Mistral, Google's Gemma, and Alibaba's Qwen. Ollama does not train models. It packages them so they run with one command instead of a day of dependency wrangling.

Two things make it popular. First, it hides the ugly parts: GPU detection, quantization, memory mapping, and prompt templating all happen for you. Second, it runs the same way on macOS, Windows, and Linux, so a command you learn on one machine works on the next.

Ollama runs the model. It does not give the model any knowledge about you. That distinction becomes important later.

Why run a model locally at all

Cloud AI is convenient, so the case for local needs to be concrete. Here is where running your own model earns its keep.

• Privacy. Your prompts and documents stay on your disk. For anything involving client data, medical notes, legal drafts, or unreleased code, this is often the deciding factor.
• Cost. There is no per-token bill. Once the model is downloaded, you can run it a thousand times for the price of electricity.
• Offline. It works on a plane, in a basement, or on a locked-down network with no internet.
• Control. You pick the exact model and version, and it does not change under you when a provider ships a new release.
• Tinkering. You can swap models, compare outputs, and script against a stable local API without rate limits.

The honest tradeoffs: a model you can run on a laptop is smaller and less capable than a frontier cloud model, first-token latency depends on your hardware, and you are responsible for updates. For drafting, summarizing, classifying, embeddings, and private Q and A, a local 7B to 14B model is often more than enough.

How to download and install Ollama

Installation takes one download or one command. Pick your platform.

macOS

1. Go to the official download page at ollama.com/download and grab the macOS build, or install via Homebrew with brew install ollama.
2. If you downloaded the app, open the .dmg and drag Ollama to Applications, then launch it once. It runs as a menu-bar background service.
3. Confirm it works: open Terminal and run ollama --version.

Apple Silicon (M1 and later) is the sweet spot here because the unified memory feeds the GPU directly, so models run fast without a discrete graphics card.

Windows

1. Download the Windows installer from ollama.com/download.
2. Run the .exe. It installs Ollama as a background service and adds it to your PATH.
3. Open PowerShell or Command Prompt and run ollama --version to confirm.

An NVIDIA GPU with recent drivers gives a large speedup, but Ollama will fall back to CPU if you do not have one.

Linux

1. Run the official install script:

curl -fsSL https://ollama.com/install.sh | sh

1. The script sets up a systemd service. Start or check it with systemctl status ollama.
2. Confirm with ollama --version.

For NVIDIA GPUs, install the matching CUDA drivers first so Ollama can use the card.

Tip: after installing, ollama runs a background server that stays up. You do not need to start it manually before running a model. If a command ever says it cannot connect, run ollama serve in a spare terminal.

Run your first model

With Ollama installed, one command pulls a model and drops you into a chat.

ollama run llama3.2

The first run downloads the weights (a few gigabytes), then gives you a prompt. Type a question, get an answer, and type /bye to exit. Next time the same command starts instantly because the model is cached.

A few commands worth knowing on day one:

• ollama pull mistral downloads a model without starting a chat.
• ollama list shows every model you have downloaded.
• ollama rm llama3.2 deletes a model to free disk space.
• ollama ps shows which models are currently loaded in memory.

Each model page on the Ollama library lists size variants. A tag like llama3.2:1b is a small, fast version, while :70b is far larger and needs serious hardware. Start small, then size up once you know a model does what you need.

Which model should you pick

The right model depends almost entirely on your RAM and, if you have one, your GPU's VRAM. Bigger models are smarter but slower and hungrier. This table is a practical starting point, not a hard rule, since quantization changes the exact footprint.

That last row is a different kind of model. nomic-embed-text does not chat. It converts text into embeddings, which are the numeric fingerprints that power semantic search over your own notes and documents. If you plan to build private search or feed a local knowledge base, you will want an embedding model alongside a chat model.

Wiring Ollama into other tools

Running a model in the terminal is fun for about a day. The real value shows up when other apps call it. When Ollama is running, it serves a local API at http://localhost:11434. Any tool that speaks that endpoint can use your local model exactly like a cloud one.

A minimal request looks like this:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Summarize the benefits of local AI in two sentences."
}'

That single endpoint is why Ollama shows up as the local-model option inside so many desktop apps, note tools, and coding assistants. You install Ollama once, then point everything else at it.

This is also where local AI starts to feel limited in an honest way. The model runs, it answers, but it has no idea who you are, what you are building, or what you decided last week. A local model with no context is a very capable stranger. If you dictate a lot of that context by voice, a tool like Contextli can capture it as you speak instead of forcing you to type. But capturing raw text is only half the job. The other half is turning it into structured, current context the model can actually use.

Where a raw local model stops being enough

Here is the gap almost everyone hits. You get Ollama running, you ask it to help with your work, and the output is competent but generic. That is not the model being weak. It is the model working blind. Output quality is capped by input quality, and a fresh model has zero input about you.

The usual patches fall short:

• Pasting context into every prompt is manual, and you forget half of it.
• A long system prompt goes stale the moment a price, a decision, or a project detail changes.
• A static notes vault helps, until you stop maintaining it, which is usually week three.

What a local model actually needs is a second brain that stays current on its own and serves that context to the model on demand. That is the idea behind Locul, a local-first desktop app that builds a searchable second brain from what you already produce, then exposes it to your AI tools over MCP. It uses local embeddings through Ollama with nomic-embed-text, so the same setup you just installed powers private, on-device search of your own knowledge.

The point that matters for this article: Ollama gives your machine a brain that can think. It does not give that brain anything to think about. Pairing a local model with context that maintains itself is what closes the gap between generic answers and answers that sound like they came from someone who knows your work. If you care about the memory side of this, the pattern is covered in depth in how to give your AI a memory that lasts, and the reusable, shareable version of that context is explained in what a memory pack is.

---

Ollama vs a full local AI stack

To make the boundary concrete, here is what Ollama handles and what still sits on you.

Ollama is the engine. The rest of the list is the fuel and the steering.

---

FAQ

Is Ollama free?

Yes. Ollama is free and open source under the MIT license, and there is no account or subscription. The models it runs are also free open-weight models. Your only real costs are disk space for the weights and the electricity to run them.

What are the hardware requirements for Ollama?

You can run small models (1B to 3B) on a modern laptop with about 8GB of RAM and no dedicated GPU. Mid-size models (7B to 8B) want roughly 8GB free and run much faster with a GPU. Apple Silicon Macs handle this well thanks to unified memory. For 14B and larger, plan on 16GB or more and ideally a discrete GPU with plenty of VRAM.

Does Ollama send my data to the cloud?

No. Ollama runs models entirely on your machine, and prompts and responses stay local. The only network activity is downloading model weights and checking for updates. If you connect Ollama to another app, that app determines whether anything leaves your computer, so review what you wire it to.

Can I use Ollama with local embeddings?

Yes. Pull an embedding model such as nomic-embed-text and call the embeddings endpoint to turn text into vectors. This is how you build private semantic search over your own notes. Locul uses exactly this approach, running local embeddings through Ollama so your knowledge base stays on-device.

How do I update Ollama and my models?

Update the Ollama app itself by re-downloading the latest build from ollama.com/download or running your package manager's upgrade command. To update a model, run ollama pull <model> again to fetch the newest published version. Run ollama list to see what you have and their sizes.

Why does my local model give generic answers?

Because it has no context about you. A raw model knows language and general facts but nothing about your projects, decisions, or preferences. The fix is not a bigger model, it is feeding it real, current context. A self-maintaining second brain like Locul solves this by building that context from what you already produce and serving it to the model. You can start free with 500 memories and local AI, no credit card, and add managed models later from the pricing page if you want them.

---

Get Ollama installed, run your first model, and you have a private, no-cost AI engine on your own machine. When the answers feel generic, that is your signal that the model needs context, not a bigger download. Pair it with a second brain that keeps itself current and you close the gap. Download Locul to give your local model something worth thinking about.