One of the UK’s top five pharmaceutical companies has introduced a private AI knowledge service that allows employees to interrogate policies, procedures, standards and operational knowledge from within Microsoft Teams.
Catapult CX built the permission-aware, source-grounded service, which is now live with employees. The challenge was not simply to create an AI interface. It was to ensure the system could retrieve useful information without bypassing existing permissions, exposing restricted data or presenting unsupported guidance as fact.
This is what private AI looks like in practice. Control must extend beyond where the model runs to the data it can access, the sources it can use, the people it can serve and the way its outputs are governed.
For regulated organisations, these controls matter at least as much as the choice of large language model.
The private AI challenge was controlled access to organisational knowledge
The organisation did not lack information. It had large volumes of it distributed across SharePoint repositories, archives, OneDrive folders and other document stores.
Policies, procedures, governance frameworks, technical standards and operational guidance all existed, but employees often had to search across multiple repositories to find the information they needed.
AI created an obvious opportunity – employees could ask questions and receive relevant answers rather than manually searching through documents.
However, a generic large language model does not understand an organisation simply because it understands language.
The case study provides a straightforward example. Asked how to structure an ‘NDA’, a generic model may reasonably interpret the acronym as a Non-Disclosure Agreement. Within a pharmaceutical company, however, it could refer to a New Drug Application.
Without access to the right organisational context, an apparently convincing answer can still be irrelevant or wrong. In a regulated environment, that can prove crucial.
The objective was therefore not simply to give employees access to AI. It was to give AI controlled access to the right organisational knowledge and only the knowledge each employee was authorised to use.
What Catapult CX and the pharmaceutical company built
Using AnswerVault, Catapult CX developed the ingestion, linking and indexing pipeline that connects the service to approved internal repositories.
The system retrieves relevant information from PDFs, Word documents, presentations, images and other internal material. It supplies that context to the model and identifies the sources used in its response.
Employees can ask questions within Microsoft Teams while the service applies the organisation’s existing permissions, data classifications and governance controls.
This implementation illustrates five controls that regulated organisations need to consider when developing private AI.
Private AI starts with the data boundary, not the model
Private AI is sometimes reduced to a hosting decision – cloud, on-premise or air-gapped.
That definition is too narrow.
Deployment architecture matters, but so do identity, data access, governance, auditability and the operating controls surrounding the system. Catapult CX’s article on private AI deployment in regulated environments examines hosted private, on-premise and air-gapped models in more detail.
The practical starting point is a simpler question – what information should this AI be allowed to access and who should be allowed to retrieve it?
That question applies whether an organisation uses a model running on its own infrastructure, a tightly controlled cloud environment or an enterprise model integrated into a wider private AI architecture.
It is also central to AI data sovereignty. Data location alone does not establish sovereignty. Organisations must consider who can administer the infrastructure, which legal jurisdictions apply and whether providers or other third parties can access the data or system.
Choosing a model without defining these boundaries solves only part of the problem.
Control 1. Make existing permissions part of the private AI architecture
An AI assistant must not become a new route around the controls already protecting organisational information.
In the pharmaceutical project, the service was designed so that users could retrieve only information they were authorised to access. Sensitive and restricted datasets remained unavailable. The case study also describes ISO/IEC 27001-aligned security controls, data-classification rules and secure development practices.
The principle sounds straightforward. Implementing it across multiple repositories is not.
The organisation must preserve the relationship between the user, the source and the permission to access it.
| Security question | Required control |
|---|---|
| What can the AI access? | Defined and approved data sources |
| Who can retrieve the information? | Existing identity and access permissions |
| What must remain unavailable? | Classification and exclusion controls |
| What happens when permissions change? | Access changes must flow through to retrieval |
| Can activity be investigated later? | Appropriate logging and audit evidence |
This aligns with ICO guidance on security and data minimisation in AI, which recognises that using AI to process personal data can change an organisation’s security risk profile.
Private AI cannot therefore be separated from identity and access management.
If a user cannot open a source document, the AI must not reveal its contents on that user’s behalf.
Control 2. Ground answers in approved organisational knowledge
Permissions determine what the system can retrieve. The next question is what information it should use to construct an answer.
The pharmaceutical company addressed this using AnswerVault, an AI knowledge assistant that uses retrieval-augmented generation, or RAG.
When an employee asks a question, AnswerVault retrieves relevant information from approved internal sources and supplies it to the model as context. It then identifies the source material used in the response.
This matters for two reasons.
- The model receives organisational context that it would not otherwise possess.
- Its answers can be traced to identifiable internal material rather than presented as unsupported model knowledge.
Security and compliance teams can therefore ask two questions:
- ‘Is this answer correct?’
- ‘Which approved information was this answer based on?’
RAG does not eliminate hallucination. Retrieval can fail, source documents can be incomplete or outdated and the model can still produce an inappropriate response.
The control is not simply to use RAG. It is to combine controlled retrieval with source attribution, evaluation, monitoring and appropriate failure behaviour.
Control 3. Design private AI to fail safely
One of the most important features of the pharmaceutical implementation is also one of the simplest.
The system can be configured to prioritise internal knowledge. If it cannot find reliable information in an approved source, it can state that it does not know rather than fabricate an answer.
That is an important distinction between general-purpose AI and AI designed for regulated work.
If an employee asks for the company’s official data-retention policy, a convincing summary of general industry practice could be more dangerous than no answer at all.
Depending on the question, the appropriate response may be to:
- retrieve the approved internal policy
- identify the relevant source
- direct the user to an accountable person; or
- state that reliable information cannot be found
In regulated AI, uncertainty should be an engineered outcome rather than something the model is encouraged to conceal.
Control 4. Put AI inside the workflow without bypassing governance
A secure system that employees avoid using will deliver limited value.
Microsoft Teams was already central to how the pharmaceutical company worked. Catapult CX therefore embedded the AI knowledge service into Teams for individual and group use rather than introducing another standalone interface.
The significance is not simply that Teams is convenient. It demonstrates that workflow integration and governance do not have to work against each other.
Employees can use AI in a familiar environment while identity permissions, approved data sources, retrieval controls and governance continue to operate underneath.
This is more effective than expecting employees to follow a separate compliance process whenever they use AI. Controls work best when they are built into the workflow rather than added through policy and approval processes alone.
Control 5. Turn static policies into active compliance support
Policies and governance documents usually sit passively in repositories. Employees must know that they exist, know where to find them and interpret them correctly.
The AI service allows employees to ask questions such as:
- Which procedure applies in this situation?
- At what point does the governance policy require approval?
- Which document covers data retention for this project?
The system retrieves the relevant material and directs the employee to the supporting sources.
This brings governance closer to the point at which work takes place. It does not mean transferring regulatory decisions to an LLM.
AI helps employees locate, understand and apply approved documentation, while accountable human review remains wherever the process requires it.
Private AI should make compliance information easier to use. It should not remove accountable people from regulated decisions.
Private AI is not limited to on-premise or air-gapped deployment
The strictest architecture is not automatically the right one.
Private AI can be deployed within a controlled cloud environment, on-premise infrastructure or a fully air-gapped system. Each model brings different security, maintenance, resilience, cost and operational requirements.
The pharmaceutical deployment shows that regulated organisations must engineer usefulness and control together.
The model is only one component. The wider operating model combines enterprise AI services, open-source components or commercial large language models while retaining flexibility over model choice. The appropriate combination depends on the workload, required data boundary and deployment constraints.
For some workloads, an approved hosted environment may provide sufficient control. Others may require on-premise infrastructure. A smaller subset may justify the additional cost and operational complexity of full air-gapping.
The important question is not; what is the most private architecture we could deploy?’ but rather, ‘what level of isolation and control does this workload require?’
- For a fuller comparison, see Catapult CX’s guide to private AI deployment in regulated environments.
Seven questions regulated organisations should answer before deploying private AI
The technical details will differ between organisations, but the control questions remain broadly consistent.
Before moving a private AI use case into production, answer these seven questions.
1. Which data can the AI access?
Define and approve the permitted data sources before designing the user experience.
2. What must remain outside its reach?
Identify sensitive, restricted or irrelevant information and establish explicit exclusion controls.
3. How will existing permissions be enforced?
The AI system should inherit or integrate with organisational identity and access controls rather than creating a parallel security model.
4. Can important answers be traced to trusted sources?
For knowledge-intensive regulated use cases, source attribution should be part of the architecture rather than an optional interface feature.
5. What happens when reliable information cannot be found?
Define how the system should refuse, qualify or escalate a request before it is deployed.
6. Which decisions still require accountable human review?
Human oversight should reflect the risk and consequences of the decision. It should be designed into the process rather than added after the system has gone live.
7. How will the system be governed throughout its lifecycle?
Security and governance do not end at deployment.
The NCSC’s Guidelines for Secure AI System Development cover secure design, development, deployment, operation and maintenance. Once a system is live, controls must continue to address areas including logging, monitoring, incident management, updates and changes in behaviour.
The UK Government’s AI Cyber Security Code of Practice takes a similar lifecycle approach, setting out baseline cyber-security principles for organisations developing and deploying AI systems.
For pharmaceutical organisations, the MHRA’s Guidance on GxP Data Integrity reinforces the importance of effective data governance throughout the pharmaceutical lifecycle.
That does not make every AI system or source document subject to GxP requirements. It does demonstrate though, why provenance, access, auditability and control must be established for any AI use case that could affect a regulated process.
Private AI is an operating model, not a hosting label
The central lesson from this pharmaceutical deployment is not that regulated organisations need another AI product.
It is that useful AI and controlled AI must be engineered together.
The model is only one component. The wider operating model combines:
- Data boundaries
- Identity
- Controlled retrieval
- Source attribution
- Safe failure
- Human accountability
- Lifecycle governance
Leave out any one of these elements and running an LLM on private infrastructure will not solve the underlying risk.
The pharmaceutical implementation shows how these controls can move beyond architecture diagrams and policy documents to become part of an AI service that employees can use in their everyday work.
Download the AnswerVault case study to see how source-grounded AI can provide controlled access to organisational knowledge.
For organisations assessing their architecture, hosting and governance options, Catapult CX’s Private AI service helps identify and deliver an approach matched to the workload, risk and required data boundary.
Frequently asked questions about private AI in regulated environments
Is private AI the same as on-premise AI?
No. On-premise infrastructure is one way to deploy private AI, but the terms are not interchangeable.
Private AI can also operate within a controlled cloud environment. What makes the system private is the combination of its data boundaries, access controls, deployment architecture, governance and operating processes.
Can private AI use a commercial LLM?
Yes. Private AI does not require an organisation to build its own model or host one entirely on organisation-owned hardware.
The architecture can incorporate commercial or open-source models, provided the deployment, data flows, access controls and governance meet the organisation’s security, privacy and regulatory requirements.
The right model and deployment approach depend on the workload, data sensitivity and risk.
How is private AI different from data residency?
Data residency describes where data is physically stored or processed. Private AI is a broader operating model covering who can access the data, how it is used, which systems and providers can process it and how the service is governed.
Keeping data in a particular country or region does not, by itself, make an AI system private or secure.
How do you prevent an AI system from accessing restricted documents?
Integrate retrieval with the organisation’s existing identity, permissions and data-classification controls.
The AI should retrieve only information the requesting user is authorised to access. Changes to a user’s role or permissions must also flow through to the AI system so that it does not continue operating with outdated access rights.
Does RAG prevent AI hallucinations?
No. Retrieval-augmented generation can ground responses in selected source material and make answers more traceable. However, retrieval can fail, source documents can be incomplete or outdated and the model can still generate an inappropriate response.
RAG should therefore be combined with source attribution, evaluation, monitoring, refusal behaviour and appropriate human oversight.
What should regulated AI do when it cannot find a trusted answer?
Where accuracy matters, the system should not fill the gap with plausible-sounding general guidance.
Depending on the use case, it should state that reliable information could not be found, direct the user to an approved source or escalate the question to an accountable person.
What activity should a private AI system log?
Logging should be proportionate to the workload and regulatory requirements. It may need to record:
- who accessed the system
- which data sources were retrieved
- the permissions applied
- the model and configuration used
- the answer and supporting sources
- refusals, escalations and errors; and
- administrative changes affecting the system
Logs should provide useful audit evidence without creating an unnecessary new store of sensitive information.
When does private AI require human oversight?
Human oversight is required where an AI-assisted decision could create material regulatory, legal, safety, ethical or operational consequences.
The organisation should define which activities AI can support, which outputs require review and who remains accountable for the final decision. These responsibilities should be established before deployment, not left for individual employees to interpret.
