An AI assistant for your company documents: how it works and where it fails

Every company has the same problem: years of documents, contracts and manuals in which nobody can find anything. A technology known as RAG (retrieval-augmented generation) turns them into an assistant that answers in its own words and cites the source. Provided it’s built properly.

How it works

Documents are split into smaller parts and stored so they can be searched by meaning, not just by words. A user’s question finds the relevant passages, and a language model composes an answer from them with references to the source. So the assistant doesn’t make things up — it quotes your documents. It’s one of the typical tasks of AI automation for business.

What drives the cost

The budget is determined mainly by the volume and quality of the documents, the number of areas the assistant should cover and the connection to permissions within the company. A sensible start is a pilot on one area, for example contracts or technical documentation. Running costs then depend on the number of questions and are in the order of hundreds of euros a month for the model’s computing power.

Three mistakes that bury the project

1. 01Pouring everything into the system, including outdated versions of policies. The assistant then confidently quotes a document from five years ago.
2. 02Not evaluating anything. Without a set of test questions, you don’t know whether it answers correctly 60% or 95% of the time.
3. 03Ignoring permissions. The assistant must not reveal the managing director’s contract to the payroll accountant.

Where search ends and action begins

On its own, the assistant answers questions. The real value comes when it can also act. “Find me the payment terms” becomes “draft an amendment based on them”; “what’s the status of the order” becomes “send the customer an update”. At that point the assistant gets access to other systems, and the emphasis on checkpoints and permissions grows. It’s sensible to start with answers and add actions only once the company trusts the assistant.

Where your data goes

The most common concern isn’t accuracy but privacy: “Are we sending years of contracts to someone else’s model?” No. The documents and the search index stay in your infrastructure or in an EU cloud, only the passage needed for a given answer goes to the model, and sensitive data can be anonymised before it is sent. A data processing agreement under Art. 28 GDPR is part of the collaboration from the start.

A well-built assistant saves dozens of hours of searching a month and backs every answer with a source. A poorly built one is an expensive generator of confident nonsense. The difference isn’t in the model but in the data and the measurement.