Private LLM is a phrase used to mean at least five different arrangements, and they do not offer the same guarantees. Because it is a marketing term rather than a technical one, the only reliable way to understand a claim is to ask what specifically is private about it.
Three separable things can be private: where the model runs, who else uses that instance, and what happens to the data you send. A given offer may deliver one, two or all three, and vendors are not always precise about which.
The five arrangements
Enterprise tier of a commercial service
The most common thing meant, and the least private in an infrastructure sense. The model is shared, run by the provider, and reached over the internet. What makes it private is contractual: an agreement that your content is not used for training, with stated retention and access terms.
This is a legitimate arrangement and appropriate for a great deal of enterprise content. It is worth being clear about what it is, though: the control is the agreement, not the architecture, and its strength is the strength of the terms.
Private endpoint in your cloud tenant
The same commercial models, reached through an endpoint dedicated to your organization inside a cloud you already use. Traffic does not cross a shared public service, network controls are yours, and the identity model is the one your other workloads use.
The model itself is still the provider's and still shared at the infrastructure level. What you gain is a controlled network path and integration with your existing controls, which for most organizations is the meaningful part.
Dedicated capacity
Model serving on compute reserved for you. No other customer's requests run on that capacity, and you generally choose the region and control when the model version changes.
That last property is underrated. On a shared service the model can be updated beneath a workflow you validated. Dedicated capacity usually means you decide when to move, which matters where output has been reviewed and signed off.
Self hosted open weights model
A model whose parameters have been published, running on infrastructure you control. Nothing leaves your boundary, you choose when anything changes, and there is no provider agreement in the path.
You also take on everything: hardware, serving, updates, evaluation, and staff who understand all of it. And the weights being published is not the same as the model being open source, which is a distinction with real consequences covered in open source and proprietary models compared.
Trained or heavily adapted in house
Rare, expensive, and usually unnecessary. Training a capable general model from scratch is beyond almost every organization outside a handful of labs. What is occasionally meaningful is substantial adaptation of an open weights model for a narrow domain, and even that is a bigger commitment than it appears, as retrieval and fine tuning compared sets out.
The question to ask instead
Rather than asking whether something is a private LLM, ask five direct questions. The answers distinguish the arrangements above without anyone having to agree on terminology.
- Where does inference physically happen? Your hardware, your tenant, reserved capacity, or shared infrastructure.
- Who else's requests run on that instance? Nobody, or other customers of the same provider.
- What is retained, by whom, and for how long? Including retention for abuse monitoring, which is common and is not the same as retention for training.
- Is our content used to train or improve the model? For the specific tier being bought, and for every feature within it.
- Who decides when the model changes? You, the provider, or the provider with notice.
An arrangement that answers all five acceptably is private enough for the purpose, whatever it is called. An arrangement whose vendor cannot answer them is not a candidate yet.
Where the ambiguity does real harm
Because the phrase covers five arrangements, it lets two people agree on a requirement while meaning different things, and the disagreement surfaces late.
The pattern is consistent. A board or a client asks for assurance that the organization is using private AI. Somebody procures an enterprise tier, which is a defensible answer. Later it emerges that the questioner meant nothing leaves our infrastructure, and the arrangement in place does not deliver that. Nobody misled anyone; the term did the work of hiding a real difference.
It runs the other way too. Organizations build self hosted environments at considerable cost because private AI was written into a requirement, when the actual obligation could have been met by a contractual term and a private endpoint. That is an expensive way to discover the requirement was never examined.
The remedy is to write down which of the five questions below the requirement is actually about, at the moment the requirement is set. It takes one paragraph and prevents both failures.
What private does not settle
The model is one component, and the arrangement around it decides most of the risk. A self hosted model connected to a retrieval layer that ignores your permissions will disclose more than a commercial service that was integrated carefully. NIST's Zero Trust Architecture makes the general point that location does not confer trust, and it applies to model serving as much as to anything else.
Two further things privacy of the model does not fix. It does not make output correct: NIST's term for confidently stated false content, confabulation, applies to a model on your own hardware exactly as it does to a hosted one. And it does not reduce the need for logging, review and access control, all of which become your responsibility rather than disappearing.
The capability question, which is usually decisive
Open weights models have improved substantially and are entirely capable for many enterprise tasks, particularly the narrow ones: classification, extraction, summarization of supplied material, structured output. For the hardest reasoning work the leading commercial models generally remain ahead, though the gap moves and any specific claim about it dates quickly.
The useful approach is to evaluate on your own tasks rather than on published benchmarks. A model that is adequate for the work you actually have is adequate, and a model that leads a leaderboard on tasks unlike yours has told you very little.
What self hosting actually requires
Organizations underestimate this consistently, and the shortfall is rarely hardware.
It requires people who can operate model serving: capacity, availability, upgrades, evaluation when a version changes, and being on call. It requires a decision about who is responsible when it is slow or wrong at four in the afternoon on a Friday. And it requires that responsibility to persist after the person who set it up has moved on.
Where that capability exists, self hosting is a sound choice. Where it does not, a private endpoint or dedicated capacity delivers most of the control with a fraction of the operational burden, which is why it is where most organizations that examine the question honestly end up. The deployment comparison in on premises and cloud AI works through the trade in full.
A reasonable position
For most organizations: a private endpoint or dedicated capacity with an agreement that forbids training on your content, with the sensitive material handled through retrieval rather than sent into training, and self hosting reserved for the categories where an obligation genuinely requires it.
That is not the most private arrangement available. It is the one most organizations can actually operate well, and an environment operated well is safer than a more private one operated badly. If that conclusion is unwelcome, the useful response is to establish which of the five questions above your obligation actually turns on, because it frequently turns on fewer of them than assumed.
Where to go next
The licensing distinctions behind open weights models are in open source and proprietary models compared. Where an environment should run is on premises and cloud AI, and how the whole environment fits together is how to build a private AI environment.
LABUSA advises on this choice and implements the result as private AI deployment work. Get in touch if you would like the five questions above answered for a specific offer you are evaluating.
Sources and further reading
- National Institute of Standards and Technology, Zero Trust Architecture, SP 800-207, August 2020.
- National Institute of Standards and Technology, AI Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, July 2024.
- Open Source Initiative, The Open Source AI Definition, version 1.0, for the distinction between published weights and open source.
Every source above was opened and read on 20 August 2026.