Clean knowledge before you build an AI agent

What shipped

Many SME teams are now being told they can build their own AI agents using tools such as Microsoft Copilot Studio, Google AI Studio and similar low-code platforms.

That is true, up to a point. A non-technical team can build useful assistants for simple tasks such as answering internal questions, drafting replies, finding the right policy, or helping a colleague follow a process.

But the first blocker is often not technical skill. It is messy knowledge.

If the agent is connected to old policies, duplicate templates, half-finished procedure notes and folders full of past versions, it will give weak answers. It may sound confident while using the wrong source. That is not a tool problem. It is a knowledge housekeeping problem.

The useful first step is smaller than most teams expect: agree a clean, current and approved set of documents before anyone builds the agent.

For Artellis, this sits squarely in agent-building training. The aim is not to turn your staff into software developers. It is to help them understand what an agent can safely do, what it should never decide on its own, and how to prepare the company knowledge it will use.

Every output should remain decision support. A person reviews, tests and approves it before operational use. The named human review step I recommend is Document Owner Approval, where the person responsible for a policy, process or template confirms that it is current and safe to use.

Who should care

This matters if you manage training, operations, customer service, HR, finance, sales support or any team that answers repeat questions from documents.

Typical examples include:

• An HR team that wants an agent to answer staff policy questions
• A customer service team that wants faster draft replies from product and warranty notes
• A finance team that wants help explaining invoice, credit control or expenses procedures
• An operations team that wants a simple assistant for standard operating procedures
• A sales support team that wants proposal language grounded in approved material

The risk is that people start with the agent builder because it looks interesting. They upload a folder, ask a few test questions, get mixed results and decide the tool is unreliable.

Often the better conclusion is simpler: the source material was not ready.

Richard recently completed a cross-departmental Copilot Studio workshop for a large organisation. The same pattern showed up there as it does in smaller Irish firms. People pick up the mechanics quickly enough when the training is practical. The harder and more valuable work is deciding what knowledge the agent is allowed to use, who owns it, and how it will be reviewed.

That is encouraging for an SME. You do not need a data science team to make progress. You do need a clear scope, a tidy knowledge set and a review habit.

A worked example

Take an Irish SME with a busy customer service team. The team wants to build an AI agent that helps staff draft answers to common customer queries.

At first, the manager points to a shared drive. It contains product guides, price notes, returns wording, warranty documents, old email replies, supplier updates and a few PDFs from previous years.

A non-technical team could connect an agent to that drive. It would probably produce answers. Some would be useful. Some would be out of date. Some might mix current policy with old wording.

A cleaner approach starts with a short knowledge preparation exercise.

First, list the top questions the agent should help with. For example, returns, damaged goods, order status, warranty limits and account changes.

Second, find the smallest number of documents needed to answer those questions. This might be one approved returns policy, one current warranty guide, one product FAQ and one set of response rules.

Third, remove or park anything that is not current. Do not ask the agent to choose between old and new versions. That is a human job.

Fourth, assign an owner to each document. If nobody owns a document, it should not be used as a trusted source.

Fifth, run the Document Owner Approval step. The owner checks that the document is current, complete enough for the first version, and suitable for internal decision support.

Only then does the team build the first agent.

In training, the team can learn how to create the agent, connect the approved material, write test questions, check citations or source references where the tool supports them, and record failures. They can also learn when to stop the agent from answering and route the query to a person.

The result is not a magic customer service robot. It is a controlled drafting assistant. Staff still review the answer, apply judgement and send the final response. That is the right level of ambition for a first internal agent.

This is also why clean knowledge is a training issue, not just a file management issue. The team needs to understand that an agent is only as useful as the material and rules it is given. A previous Artellis insight on practical SME adoption is available in the insights library.

Where I'd start

Start with one workflow, one team and one narrow question set.

Do not begin by asking, “What can we automate?” Ask, “What repeat questions do we already answer from documents?” That question keeps the project grounded.

Then create a simple knowledge pack:

• The task the agent will support
• The questions it should answer
• The questions it must not answer
• The approved documents it can use
• The named owner for each document
• The Document Owner Approval check
• The human review step before any answer is used operationally

For many SMEs, this can be done before a training session. It makes the training far more useful because staff are learning on their own material, not a generic demo.

If you are planning agent-building training, resist the urge to gather every document “just in case”. A smaller, current, approved set will usually beat a larger, messy one.

The clear takeaway: before you teach a team to build agents, teach them to choose and approve the knowledge those agents will rely on.

If that is where your team is, book a 20-minute call about agent-building training and use it to pick the first workflow, the first document set and the review step.