Run an LLM locally on your laptop
You can run an LLM locally with no server, no account and no data leaving the machine. Pick a model that fits your memory, pull it from Hugging Face with one command, and give Locul a private brain that works offline.
Quick answer
To run an LLM locally, install Ollama and run ollama run hf.co/unsloth/Qwen3.5-9B-GGUF:Q4_K_M. That 5.68 GB model suits a 16 GB laptop; on 8 GB use Qwen3.5 4B, and on 32 GB or more try Qwen3.8 27B. In Locul, open Settings and use Find a model to search Hugging Face, see what fits your RAM and download it in one click. Prompts never leave your machine.
Which LLM can I run locally with my RAM?
With 16 GB of memory you can run a 9 to 12 billion parameter model comfortably; 8 GB fits a 4B model, and 32 GB or more opens up the 27 to 35B class.
The rule we use in the Locul app: a model needs its file size plus 20% (at least 1.5 GB) for the context and the runtime. If that is under 75% of your RAM it fits, up to 90% it is tight, and above that it is too big. On a Mac, RAM and graphics memory are the same pool, so this is the number that matters.
| Your RAM | Best pick | File | Also fits |
|---|---|---|---|
| 8 GB | Qwen3.5 4B | 2.74 GB | Qwen3.5 9B is tight; Gemma 4 E4B (4.98 GB) is tight |
| 16 GB | Qwen3.5 9B | 5.68 GB | Gemma 4 12B (7.12 GB); Qwen3.5 9B at Q8 (9.53 GB) |
| 24 GB | Gemma 4 12B | 7.12 GB | Qwen3.8 27B (16.46 GB) and Gemma 4 26B-A4B (16.95 GB) are tight |
| 32 GB | Qwen3.8 27B | 16.46 GB | Gemma 4 26B-A4B; Qwen3.6 35B-A3B (22.13 GB) is tight |
| 48 GB or more | Qwen3.6 35B-A3B | 22.13 GB | Qwen3.8 27B at Q8 (29.05 GB) |
Models with "A3B" or "A4B" in the name are mixture-of-experts: only about 3 or 4 billion parameters work on each word, so they answer faster than their size suggests, but the whole file still has to fit in memory.
How do I run an LLM locally with Ollama?
Install Ollama, then run one command that names a model on Hugging Face; Ollama downloads it and starts a chat.
Install Ollama
On a Mac or Windows PC, download the app from ollama.com/download and open it once. On Linux:
bash$ curl -fsSL https://ollama.com/install.sh | shRun a model from Hugging Face
Any GGUF repository on Hugging Face works. The part after the colon picks the quantisation:
bash# 16 GB laptop: Qwen3.5 9B $ ollama run hf.co/unsloth/Qwen3.5-9B-GGUF:Q4_K_M # 8 GB laptop: Qwen3.5 4B $ ollama run hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M # 32 GB laptop: Qwen3.8 27B $ ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_MLeave the tag off and Ollama picks Q4_K_M when the repository has it, which is the right default for most laptops.
Check what you have
bash$ ollama list $ ollama ps
"By default, the Q4_K_M quantization scheme is used, when it's present inside the model repo."
Hugging Face docs, Use Ollama with any GGUF model on the Hub
What does it look like on a real laptop?
This is the unedited output from a 16 GB M1 Pro MacBook running Ollama 0.34.0 on macOS, pulling a small model to check the wiring.
$ ollama pull hf.co/unsloth/Qwen3-0.6B-GGUF:Q4_K_M
pulling manifest
pulling ac2d97712095: 100% ▕████████████████▏ 396 MB/396 MB 4.8 MB/s
verifying sha256 digest
writing manifest
successThen the same machine answered through the OpenAI-compatible API that Locul and other apps use:
$ curl http://localhost:11434/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"hf.co/unsloth/Qwen3-0.6B-GGUF:Q4_K_M","messages":[{"role":"user","content":"Draft a reply to Priya..."}]}'We used a tiny 0.6B model here only to prove the path works end to end. For real office work, pick from the RAM table above. Afterwards we removed the test model with ollama rm.
"The client requires an API key value, but Ollama ignores it."
Ollama docs, OpenAI compatibility
How do I connect Locul to a local model?
Open Settings in Locul, use Find a model on the local models card, and press Download; Locul talks to Ollama for you.
Open the local models card
In Locul, go to Settings. If Ollama is missing, the card offers Get Ollama; if it is installed but stopped, it offers Start it.
Choose Find a model
Search Hugging Face from inside the app. Each quantisation shows its size and a label for your machine: fits, tight or too big, worked out with the rule above.
Download
One Download button pulls the model into Ollama. When it finishes, Locul uses it for chat.
Optional: Advanced
Under Advanced you can switch the chat provider to LM Studio, pick a different chat or embedding model, or set the Ollama address if Ollama runs on another port or machine. Reset to automatic undoes it all.
Where your prompts go: nowhere. With a local model, Locul and the model both run on your laptop, so prompts and memory never leave it. That makes this the safest path for HR notes, health data or anything under a confidentiality clause.
Ollama, LM Studio or llama.cpp: which local runner should I use?
Use Ollama if you want Locul to manage models for you, LM Studio if you prefer a desktop app with its own model browser, and llama.cpp if you want the bare engine.
| Ollama | LM Studio | llama.cpp (llama-server) | |
|---|---|---|---|
| What it is | Background service and CLI | Desktop app with a chat window | The C++ engine the others build on |
| Default address | localhost:11434 | localhost:1234 | localhost:8080 |
| OpenAI-compatible API | Yes, under /v1 | Yes, under /v1 | Yes, under /v1 |
| Works with Locul | Yes, managed from the app | Yes, pick it under Advanced | Not in the app today |
Comparing whole apps rather than runners? See LM Studio alternatives for teams.
What can a local model actually do for office work?
Plenty of the daily admin that you would never paste into a US chatbot, such as tidying a forwarded email about a client's lease.
The 0.6B test model on the laptop above replied:
It kept the two rules, wait for Anneke and no promise on the deposit, but it wrote to the wrong person and skipped the Dutch question and the three dates. That is what a 0.6B model does. A 9B model from the 16 GB row is the realistic floor for this kind of email, and the benchmark below will show how much better.
How good are local models on real office tasks?
We are scoring laptop-sized models on messy, real office work, and the results will appear here.
Benchmark in progress
Emails like Tom's, broken spreadsheets and contract clauses, run on 16 GB and 32 GB laptops, scored by people who do this work. Coming soon.
Frequently asked questions
Can I run an LLM locally on 8 GB of RAM?
Yes, a 4B model such as Qwen3.5 4B at Q4_K_M (2.74 GB) fits with room to spare. A 9B model is tight on 8 GB, so close other apps or move to 16 GB for daily use.
Is running an LLM locally free?
The software and the open models are free to download. You pay with your own hardware: memory, disk space and battery while the model works.
Do local models need the internet?
Only to download the model once. After that Ollama runs offline, and your prompts never leave the machine.
What does Q4_K_M mean?
It is a 4-bit quantisation format for GGUF files. It shrinks a model to roughly a quarter of its full size while keeping most of its quality, which is why Ollama picks it by default when a repository has it.
Can I use any Hugging Face model with Ollama?
You can use any repository that ships GGUF files. Run ollama run hf.co/{username}/{repository}:{quantisation}, or search from Find a model in Locul, which shows only files that suit your RAM.
Sources
Every command and quote on this page comes from these primary sources, checked on 24 October 2026.
Junaid builds Locul and sets up private AI for small regulated firms. He tests every command on this page on his own machines before it ships.