An AI assistant inherits the gaps in its knowledge base
An AI assistant needs an approved, current and appropriately accessible knowledge base before it can be relied on for useful answers. Review the source material, resolve conflicting instructions and define what the assistant should do when an answer is missing. A larger document collection is not automatically a better source.
Put the advice into practice
Build a workflow your team can actually rely on
Connect systems and follow-up with agreed rules, human review and exception handling.
Discuss your workflowA team may discover that the difficult work is not connecting a tool. It is deciding which version of a process is authoritative, who may see it and whether the information actually answers the questions people ask.
Inventory the information the assistant would use
List the proposed documents, pages and records. Identify the owner, intended audience, review status and permitted use of each source. Separate public material, internal procedures and information restricted to particular roles.
Do not assume that access to a shared folder establishes permission for an assistant to reveal every file to every user. The knowledge design needs to preserve the organisation’s access boundaries. A tool should not become an easier route around information controls that already exist.
Resolve conflicting and incomplete instructions
Inspect the questions the assistant is expected to answer. Then look for the actual source that supports each answer. If two documents disagree about an approval step, a service condition or a contact route, decide which source is current before connecting both.
The review can reveal knowledge that exists only in a person’s memory. Record that gap and ask the responsible expert to produce an approved explanation. Do not fill it with a plausible model-generated answer and treat the result as an organisational fact.
Give important sources an owner and a review trigger. A document can become stale because the service, team, system or policy changes. A date on the file is useful context, but it does not prove that the material remains correct.
Build a readiness sheet
| Check | Evidence to collect | Decision |
|---|---|---|
| Authority | Named owner and approved version | Include, revise or exclude |
| Coverage | Questions the source can actually answer | Add missing information or narrow the assistant’s scope |
| Consistency | Conflicting statements and their owners | Resolve before use |
| Permissions | Who may access the source and the answer | Apply the appropriate boundary |
| Maintenance | What change triggers a review | Assign an ongoing owner |
| Uncertainty | What happens when no supported answer exists | Provide a clear handover route |
Use the sheet to make a bounded first selection. A small set of reliable answers can be more useful for a pilot than a large collection whose accuracy and permissions are uncertain.
An illustrative internal assistant
Imagine a hypothetical consultancy wants an assistant to answer questions about starting a new project. The proposed sources include an old onboarding checklist, a current project template and several informal notes. They disagree about who approves the start date.
The team first agrees the current approval rule and updates the source. It then limits the pilot to questions about preparation and handover. Questions about commercial exceptions go to an authorised person. This creates a clearer test than asking the assistant to infer a policy from conflicting material.
No outcome in this scenario is a claim about a delivered client system. It illustrates the information decisions that should precede a technical connection.
Test supported answers and missing answers
Prepare a set of realistic questions, including ambiguous wording, an outdated assumption and a question outside the approved scope. Review whether the assistant can point to the relevant source and whether it handles uncertainty without inventing a commitment.
The National AI Centre’s AI adoption guidance calls for testing, documented acceptance criteria and ongoing monitoring. Apply those principles to the actual purpose and risks of the assistant, rather than relying on a demonstration that answers only ideal questions.
Keep the knowledge review connected to operation. When a source changes, confirm that the assistant’s approved information changes too. Use the existing chatbot testing guide for a fuller review of a public-facing conversational experience. Knowledge readiness and interface testing address different parts of the same system.
