An AI assistant that answers on internal documents, without sending your data around
Want an AI assistant that answers on your internal documents without sending your data around? Here is how to build it, what it needs, and when NOT to.
The COO almost always frames it like this: “My team wastes half an hour hunting for the right procedure in SharePoint. Can’t we just put an AI that answers?”
Honest answer: yes, you can, and it’s often worth it. But an internal knowledge base AI assistant is not “upload the PDFs into ChatGPT”. Three decisions determine whether it works or becomes yet another tool nobody uses after two weeks.
Key takeaways:
- An AI assistant on your company documents (RAG) answers employee questions by citing the exact source inside procedures, contracts, manuals and policies, without the person searching by hand.
- Your data must NOT leave your company: at Soraia no LLM is trained on client data, and you can use European infrastructure or on-premise.
- The real value is not the model, it’s order in the knowledge base: an assistant fed messy documents gives messy answers.
- Start on a narrow scope (one domain, not the whole company), with the first release in production in 4 weeks.
- Do NOT build it if you have few documents or they change every week: a good internal search is enough.
What an internal document assistant actually is
It’s an agent that, when an employee asks a natural-language question (“how many bereavement leave days do I have?”, “what is the termination clause with supplier X?”), searches inside your internal documents, finds the right passage and answers citing the source. The technique is called RAG (Retrieval-Augmented Generation): the model doesn’t “know” the answer, it retrieves it from your files and rephrases it.
The difference from ChatGPT is simple: ChatGPT knows everything except your business. This one knows only your business.
The three decisions that matter
1. Where the data lives
This is the COO’s first question, and rightly so. Your contracts and HR policies must not end up training a third party’s model.
How we handle it at Soraia:
- Documents stay in your storage. The assistant reads them at query time, it doesn’t copy them into the model.
- No training of LLMs on client data. Ever.
- European infrastructure when required, on-premise when needed.
- Article 28 GDPR DPA included in the contract, with an audit log on every answer.
This isn’t heroism: on HR documents and contracts, without these guarantees we couldn’t even start.
2. The scope, not everything at once
The classic mistake: “let’s connect everything, SharePoint, Drive, email, the ERP”. Result: the assistant pulls from contradictory sources and loses credibility by the third wrong answer.
Start from one domain: only the HR manual, or only supplier contracts, or only quality procedures. A narrow scope can be measured, corrected, and trusted by people. Then you widen it one release at a time. It’s the same principle we use for AI agents: first release in production in 4 weeks, then iterate.
3. Order in the knowledge base
This is the part nobody wants to hear: if your documents are messy, duplicated or contradictory, the assistant inherits that mess. It’s not magic, it’s an amplifier.
Often the most useful phase isn’t technical: it’s cleaning up. Figuring out which version of a procedure is the right one, archiving duplicates, flagging authoritative sources. That’s part of custom software development, because the assistant is just the top layer over an ordered base.
Where it pays off most
Three cases where we see the clearest return:
- Onboarding: new hires ask the assistant instead of interrupting colleagues. The time experts spend answering the same questions drops in a measurable way.
- Internal customer support: whoever answers clients finds the right policy in seconds instead of searching. It fits well with customer & compliance automation, where the agent suggests and a human approves.
- Operating procedures and compliance: where a wrong answer costs, citing the source is what makes the assistant genuinely usable.
When NOT to build it
We’ll say it to your face:
- Few documents (under a few hundred pages) → a good internal search is enough and costs less.
- Documents that change every week → you’ll maintain the assistant more than you use it.
- Nobody owns the knowledge base → without an owner keeping documents clean, the assistant degrades fast.
But if you have a stable corpus, searched often and important to get right, it’s one of the most readable-ROI projects out there.
Want to know if your case holds up? Let’s talk for 20 minutes or take the check-up: we’ll tell you honestly whether you need an assistant or whether fixing the search you already have is enough.
Frequently asked questions
What people usually ask us.
Do our documents end up training an external model?
How is this different from giving everyone ChatGPT?
How long until we have something usable?
What if the answers are wrong?
Keep reading
Management reporting with AI for the CFO
The management report always lands mid-month. How an AI agent prepares management control and CFO dashboards, with traceable data and a full audit log.
Automated commission calculation with an AI agent
Still calculating sales commissions by hand every month across contracts and deals? How an AI agent reads the rules and does it for you, without errors.
Next step
Where are you on the AI journey?
The check-up gives you an AI readiness score (0-100) + 3 concrete next steps. 3 minutes, no email.