Most private AI projects fail in the same way. They begin with a model, acquire infrastructure, and arrive at the questions about data and identity late, when the answers are expensive. The order below is close to the reverse, and it exists because the decisions that are hardest to change later are the ones that get made first here.
This is a description of sequence and structure rather than a deployment recipe. What a specific environment needs depends on the material it will touch and the obligations attached to it.
Start with a use case, not a capability
The first decision is what the environment is for, stated narrowly enough to be wrong. "Improve productivity" cannot be designed against. "Let the support team find answers in our own documentation without reading six systems" can.
A narrow first use case does three useful things: it bounds which data is in scope, it makes the value measurable, and it means the first thing built is small enough to be replaced when the requirements turn out to be different from what everyone said.
It is also worth deciding, at the same moment, what the environment is deliberately not for. That sentence prevents a great deal of scope drift later.
Settle the rules before the architecture
Which categories of data may be used, who is accountable, who reviews output before it is acted on, and what happens when it is wrong. These are governance questions, they are not technical, and answering them late means rebuilding the parts of the environment that assumed a different answer.
They belong with the work described under AI Governance, and the reason to settle them first is practical rather than procedural: the retrieval design cannot be specified until somebody has said which content is in scope.
Know what data is actually in scope
An inventory of the sources the environment will read, and a classification of what is in them. Not a taxonomy exercise. A short, enforceable rule about what may be indexed, what may leave the boundary and what is excluded outright.
Two things are consistently underestimated at this stage. Content in a shared drive is frequently more sensitive than anyone believes, because material accumulates without anyone re assessing it. And permissions on those sources are frequently wrong in ways nobody has noticed, because until now nothing has systematically tried to read everything at once. An AI retrieval layer is very good at finding both.
Decide how identity works
Two distinct questions, and conflating them is the most common architectural mistake in this whole sequence.
Who may use the system. Your existing identity provider, single sign on, multi factor authentication, and the same joiner and leaver process as everything else. This part is ordinary.
What the system may reach on a user's behalf. This is the part that is specific to AI, and it is where the shortcut lives. The tempting design gives the service one identity with broad access and filters afterward. It fails in ways that are hard to detect, because the failure looks like a helpful answer.
The principle is that AI must not become a route around your existing authorization model. NIST's Zero Trust Architecture states the general form: no implicit trust is granted based on where something sits. A retrieval service inside your network has earned nothing by being there.
Choose a model, and expect to change it
Later than most projects put it, and less permanently. What matters at this stage is not which model is best today but whether the environment can survive changing it: an abstraction between the application and the model, evaluation you can re run, and a decision about whether a version can change underneath you without notice.
Model choice interacts with the deployment decision covered in on premises and cloud AI, and the licensing distinctions matter more than they appear to, since terms described as open differ substantially from one another.
Design retrieval as an access control problem
This is the layer that decides whether the environment is safe, and the one most often treated as a search feature.
The requirement is that filtering happens as part of the query, evaluated against the identity of the person asking, rather than applied to results afterward. Retrieve broadly and then remove what the user may not see, and three failure modes follow: a summary generated before filtering can carry restricted content into a permitted answer, relevance ranking computed over the unfiltered set leaks information by omission, and an error path that skips the filter fails open while looking entirely normal.
Retrieved content also has to be treated as untrusted input rather than as instruction. Text pulled into a model's context is read as instruction, and a document containing an instruction is a plausible thing to find in a shared drive. The OWASP GenAI Security Project tracks this as prompt injection, alongside sensitive information disclosure and improper output handling. Our colleagues have written about the same problem from the content platform side in security and privacy for AI-powered CMS platforms, which is the worked example of this layer in a specific context.
Put the environment somewhere
Infrastructure comes after the decisions above, not before, because those decisions determine what the infrastructure has to satisfy. The available shapes and the trade between them are set out in the deployment comparison.
What belongs here is the boundary design: which networks the environment sits in, what may reach it, what it may reach outward, and where secrets live. The last of those is mundane and is the one that goes wrong, because an environment that needs credentials to six systems has quietly become a place where six systems' credentials are stored.
Integrate through supported interfaces
Connecting to the systems that hold the knowledge is what makes the environment useful, and it is where scope grows fastest. Two rules keep it manageable.
Integrate through interfaces the source system supports, so that its own permission model comes with the data rather than being reimplemented. And distinguish clearly between reading and acting. A system that can answer questions has a smaller blast radius than one that can send, change or delete, and where an action is consequential an approval step is not a failure of automation.
Apply the controls the environment inherits
Most of the security work here is not novel. Least privilege, authentication, authorization, encryption in transit and at rest, segmentation, secrets management and patching are the controls any system of this sensitivity needs, and NIST's SP 800-53 catalog is the reference for them.
The additions specific to AI are narrower than the marketing suggests: constrain what the surrounding system can do so a compromised instruction cannot trigger an action, validate output before it is used in a context where it could execute, and keep the model's permissions distinct from the user's.
Make it observable
Record who asked what, which sources were retrieved, what was answered and which tools ran. Without that, an incident cannot be reconstructed and a policy cannot be evidenced.
The log is also a new store of potentially sensitive material, which means it needs its own retention rule, its own access control and its own place in the data classification. That is a real cost and a reason to design it deliberately rather than an argument for logging less.
Plan to operate it
The environment is not finished when it works. Model versions change, sources change, permissions change, and people join and leave. Access review, a process for evaluating a model change before adopting it, and a named owner are what keep an environment correct rather than merely correct at launch.
Review on a date rather than on a rumor. An environment nobody has looked at since it was built is the normal failure, and it is a quiet one.
The layers this produces
Followed through, the sequence yields a recognizable structure: users authenticated by your identity provider, reaching an interface, which reaches a model through an abstraction, alongside a retrieval layer that queries knowledge sources under the asker's permissions, with integrations to business applications, all inside a network boundary, with logging and monitoring across it.
Drawing that diagram first is the temptation and the mistake. The diagram is an output of the decisions about use case, data and identity, and an environment built from the diagram inward usually discovers those decisions in the wrong order.
Where to go next
If you are earlier in the question, the definition of private enterprise AI sets out what is being controlled, and private and public AI compared covers whether a controlled environment is warranted at all.
If the environment already exists and the question is how to make it safe to operate rather than how to build it, secure generative AI for business covers the controls that apply to generative AI use across an organization.
LABUSA designs and secures these environments as part of our private enterprise AI capability. Most engagements start with the inventory rather than the architecture, because the inventory is usually where the surprises are. Get in touch to discuss yours.
Sources and further reading
- National Institute of Standards and Technology, Zero Trust Architecture, SP 800-207, August 2020.
- National Institute of Standards and Technology, Security and Privacy Controls for Information Systems and Organizations, SP 800-53 Revision 5.
- National Institute of Standards and Technology, AI Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, July 2024.
- OWASP GenAI Security Project, OWASP Top 10 for LLM Applications, 2025. Community guidance on threats and mitigations, not on regulatory obligation.
Every source above was opened and read on 19 August 2026.