In your building
Servers in your own comms room. Full physical control, and it can run with no internet at all. You also own the power, the cooling and the 3am failures.
Locul / Guides / On-premise LLM
On-premise LLM: what it really takes to run one for a 20 to 150 person firm.
Most guides to on-premise AI are written for engineers with a spare GPU at home. This one is for the IT manager who has to put a shortlist in front of the board: what to buy for your headcount, what the server room has to handle, who keeps it alive, and when renting a dedicated server in your region is the better answer.
The short answer
An on-premise LLM is a language model running on GPU servers your firm owns, so no prompt or document leaves your network. For a firm of 20 to 150 people that usually means one or two servers with two to four data-centre GPUs, a rack with proper power and cooling, and someone responsible for patching it.
If the reason is "client data must stay in our control" rather than "client data must stay in our building", a dedicated server in a UK or EU data centre does the same job with less to look after. Both are covered below.
The question we get most
"What do I actually need to buy?"
This is close to a real email from an IT manager at an engineering firm. It has every detail that matters: an insurer's wording, a rack, two power feeds, a hot summer and a board date. The rest of this guide answers it.
An example. The firm and people are invented.
From: Priya Raman, IT Manager
Cc: Darren Cole, Operations Director
RE: AI pilot - hardware (again)
Hi both,
Following Tuesday's call. Insurer came back and the wording is now 'no client data to be processed outside UK-controlled infrastructure' which after the March incident I think we have to read as our building, not 'a UK region' on someone's cloud. Darren disagrees, he's copied.
We have the comms room on floor 2 with one free 42U rack, 2 x 16A feeds, aircon is ok-ish in summer (it hit 29C in July, I have the logs). 85 staff, maybe 40 would use it daily. Budget line is capex not opex which helps.
What do I actually need to buy? Please not 'it depends'. A shortlist I can put to the board on the 11th would be ideal.
Thanks
Priya
Three meanings of "on-premise"
Decide which one your contract means.
Darren and Priya are both right, depending on the wording. Before buying anything, read the clause with whoever wrote it.
Your rack, their building
Hardware you own in a colocation data centre. Better power and cooling, your equipment, your keys. Usually satisfies "infrastructure we control".
Dedicated, hosted in-region
A server rented for your firm alone in a UK or EU data centre. Not shared with anyone, no overseas transfer, nothing to maintain. Often enough for "UK-controlled", worth checking with the insurer.
Hardware by team size
Size for the busiest ten minutes, not the average day.
What matters is how many people ask something at the same moment. In an office, that is roughly a quarter to a third of daily users at 9:15 on a Monday. Size GPU memory for the model plus those conversations, then add one GPU of headroom.
| Daily users | Peak at once | Model class | GPU memory | A typical build |
|---|---|---|---|---|
| 10 to 25 | 4 to 8 | Mid-size, around 30B | 48 to 96 GB | 1 server, 2 workstation-class GPUs with 48 GB each |
| 25 to 60 | 8 to 20 | Large, around 70B, or 30B with long documents | 160 to 192 GB | 1 server, 2 x 80 to 96 GB data-centre GPUs, or 4 x 48 GB |
| 60 to 150 | 20 to 45 | Large, plus a small fast model for quick jobs | 320 GB and up | 2 servers with 4 GPUs each, so one can fail without stopping work |
Priya's 40 daily users sit in the middle row. One 4-GPU server draws roughly 2 to 3 kW under load, which a 16A feed at 230V can carry, and turns almost all of it into heat. At a 29°C room temperature, the GPUs will slow themselves down on the afternoons people need them most. Fix the cooling before buying the cards.
Three ways to run it
On-premise, cloud API or a dedicated server.
| Public cloud AI API | On-premise, built in-house | Locul, on your servers or dedicated in-region | |
|---|---|---|---|
| Where data is processed | The provider's cloud | Your building | Your building, or a UK or EU server for your firm alone |
| Up-front cost | None | Hardware, plus setup time | Hardware if on your site, none if hosted |
| Who patches and monitors | The provider | Your team | Locul or a certified local partner |
| Runs with no internet | No | Yes | Yes, on your own servers |
| Permissions and audit log | Depends on the plan | Whatever you build | Built in |
| Ready-made bots for office work | No | No | Yes, and each asks before anything leaves |
Weighing the build-it-yourself route with an open-source chat app? Read the self-hosted LLM guide first, and the fair comparison with Open WebUI.
Before the board meeting
A shortlist that is not "it depends".
For a firm like Priya's, this is the order we would work through.
Get the clause read properly
Ask the insurer in writing whether a dedicated server in a UK data centre counts as UK-controlled. If it does, half this list disappears.
Fix the room
Cooling that holds the rack inlet under 27°C on the hottest day, and a UPS sized for the GPU server.
Size for peak, buy for failure
One 4-GPU server for 40 daily users. A second one only if a day without AI would stop billable work.
Name the owner
A person or a partner responsible for patches, backups and model upgrades, with time in their week for it.
Test on your own documents
A week of real questions from three teams beats any benchmark, including ours.
The Office AI Benchmark
Which open model should run on the server you buy? We test them on real office work: messy emails, spreadsheets with broken formulas, contracts with the key clause on page 31. The results are public.
On-premise LLM questions
Asked by IT managers, answered plainly.
What is an on-premise LLM?
An on-premise LLM is a large language model that runs on servers your organisation owns, in your own building or a rack you control in a colocation centre. Prompts and documents are processed on that hardware and never sent to a cloud AI provider.
How much GPU memory do we need?
Enough to hold the model and the conversations in progress. As a rule of thumb, a 30-billion-parameter model at 8-bit needs around 32 GB for the weights plus room for each active user, and a 70-billion-parameter model needs around 75 GB plus that room. The table on this page turns that into servers by team size.
Can an on-premise LLM run with no internet connection?
Yes. Once the models and software are installed, everything runs on your network. Updates are brought in on a schedule you control, by a person, through whatever route your security policy allows.
Is on-premise more secure than a dedicated hosted server?
It gives you physical control, which some insurers and contracts require. A dedicated server in a UK or EU data centre is not shared with anyone else either, and the data centre usually has better power, cooling and physical security than an office comms room. The right choice depends on what your contracts actually say.
Which open models are good enough for office work?
The leading open-weight models from the Qwen, Mistral, Llama, Gemma and DeepSeek families handle drafting, summarising and document questions well. Which one is best changes every few months, so we publish results on real office tasks in the Office AI Benchmark.
Can Locul run on our own servers?
Yes. Locul installs on GPU servers in your building, including on isolated networks, and a certified partner or our team sets it up and keeps it updated. See the deployment options.
Next step
Have Locul sized for your building.
One call. We read the clause with you, size the hardware for your headcount and show Locul running on it.
- Runs on your servers, even with no internet
- Set up and kept updated by a partner or by us
- Priced per seat, never per prompt