The model runner
Ollama is the easy start. For 20 people asking at once you usually move to vLLM or a tuned llama.cpp server, and that move is real work.
Locul / Guides / Self-hosted LLM
Self-hosted LLM for a team: build it yourself, or have it run for you.
Getting a model running on one machine takes an afternoon with Ollama and Open WebUI. Getting the same thing to serve 50 colleagues every day, with logins, permissions, backups and an audit trail, is a part-time job. Here is what the job involves, so you can decide who does it.
The short answer
A self-hosted LLM is a language model you run on hardware you control, so prompts and documents never reach an AI provider. For one person it is a free afternoon project. For a firm, the model is the easy part: the work is logins, permissions, backups, updates and an audit trail, every week.
Build it if you have someone with the time and the interest. Have it run for you if you do not, and keep the data on your own or dedicated servers either way.
What it looks like at month six
Monday, 08:12, in the IT channel.
Most self-hosted setups do not fail on day one. They fail quietly after an automatic update, a few weeks before an audit. This exchange is invented, and every part of it is common.
Karl · 08:12
so the Open WebUI box fell over again over the weekend. Docker update pulled a new image, the vector DB migration ran half way and now nobody's chats load. I've rolled back but the RAG index for the HR handbook is gone, re-embedding will take most of today. Also realised the GPU drivers on that box haven't been patched since June. Not ideal given the audit is in 3 weeks.
Karl · 08:14
before anyone asks, no I don't know who added the 'test-llama-uncensored' model. it's off now.
Sophie · 08:31
can we talk about whether we really want to be running this ourselves
The home-lab stack
Great on one machine. Where it stops scaling.
The usual open-source stack is genuinely good, and many firms start here. These are the points where a team outgrows it.
The chat app
Polished and fast-moving. Fast-moving means frequent updates, database migrations and licence terms to keep reading.
Document search
Works until a migration fails or someone asks why the answer quoted a file they should not see.
What a firm needs that a hobby stack lacks
Six things to have answers for before the audit.
| Typical DIY setup | Locul, managed | |
|---|---|---|
| Logins | Local accounts, or SSO if someone wires it up | Single sign-on available, set up with your existing accounts |
| Who can see which files | Per workspace or group, set by hand | Answers follow the permissions already on your drives |
| Audit trail | Off by default in most apps, often without chat content | Every question, answer and file used, logged on your side |
| Backups and restore | Your script, tested when it fails | Handled by Locul or your partner |
| Updates | When someone has time, or automatically and untested | Tested first, applied in a window you agree |
| Actions that leave the firm | Not built in | Every email or post waits for a person to approve it |
The real cost
The software is free. The hours are not.
Add these up for your own firm before deciding. The biggest number is rarely the hardware.
GPUs
Bought or rented, sized for your busiest ten minutes. Our on-premise guide has a table by team size.
Power and room
A 4-GPU server draws 2 to 3 kW and needs cooling to match.
Weekly upkeep
A few hours of a skilled person's week, every week, for patches, backups and questions.
The bad days
The Monday above. A lost index, a failed GPU, an audit question you cannot answer.
Build, buy or managed
Pick by who will own it.
Do it yourself
You have an engineer who enjoys this and has a few hours a week. Your data rules are simple, and an outage is an inconvenience.
Public AI business plan
A contract with a US provider is acceptable to your clients and insurer, and you do not need it to run in a specific building.
Self-hosted, run for you
The data must stay on your servers or in your region, and nobody in-house should be on call for it. This is what Locul is for.
Comparing specific tools? Read our fair comparisons with Open WebUI, AnythingLLM, LM Studio and Msty.
The Office AI Benchmark
We test open models on real office work: messy emails, spreadsheets with broken formulas, forwarded chains with the ask buried in paragraph four. See how close they get to the big cloud models.
Self-hosted LLM questions
Plain answers before you build.
What does self-hosted LLM mean?
A self-hosted LLM is a language model you run on hardware you control, instead of calling a provider's API. It can be a laptop, a server in your office, or a server you rent. The model, the prompts and the documents stay on that hardware.
What is the usual self-hosted stack?
A model runner such as Ollama, llama.cpp or vLLM, a chat interface such as Open WebUI, LibreChat or AnythingLLM, a vector database for searching your documents, and a reverse proxy with single sign-on in front. Each part is maintained by a different project on a different release schedule.
Is a self-hosted LLM safe for client data?
It can be safer than a public AI service, because nothing leaves your servers. It is only as safe as its upkeep, though: unpatched drivers, a chat app exposed to the internet or a missing permission check all undo the benefit.
How much time does a self-hosted LLM take to run?
For a team, plan for a few hours a week once it is stable: testing and applying updates, checking backups, answering staff questions, and a few days each time you move to a better model. Getting it stable takes longer.
Can Locul be self-hosted?
Locul runs on servers in your building or on dedicated servers in a UK or EU data centre, so the data stays self-hosted. The difference is that a certified partner or our team keeps it running. See the deployment options.
Next step
Keep it self-hosted. Stop being on call for it.
One call. We look at what you run today and show the same work on Locul, on your servers or a dedicated one in your region.
- Data stays on your own or dedicated servers
- Patched, backed up and upgraded for you
- Priced per seat, never per prompt