A chatbot that answers from your documentation cannot be better than that documentation. Teams often discover this late, after the build, when the bot repeats an outdated policy or has nothing to say about a common question. A few days spent on the content before the technical work starts prevents most of these problems.
Start with the questions, not the documents
Export the questions customers or staff actually ask: support tickets, chat logs, emails, the search terms used in your help centre. Group them by topic and count them. This list is the specification for the knowledge base. A topic with many questions and no article is the first gap to fill.
Check coverage
For the twenty or thirty most frequent questions, find the document that answers each one. Three things usually turn up.
- Questions with no written answer, because the answer lives in the heads of experienced staff.
- Questions answered in several places, with small differences.
- Answers that exist but are buried in a long document about something else.
The first group needs writing. The second needs a decision about which version is correct. The third needs restructuring.
Remove contradictions and old content
A human reader notices that a page looks old and discounts it. A chatbot does not. If last year's price list is still in the knowledge base, it will be quoted. Archive superseded documents, and where two sources disagree, fix or remove one. This is tedious work and it has more effect on answer quality than any technical setting.
Write sections that stand on their own
The system retrieves passages, not whole documents. A passage has to make sense without the page around it.
- Use clear headings that say what the section is about.
- Name the product, plan or region in the section itself, not only at the top of the page.
- Avoid "as mentioned above" and "see the previous step".
- Keep one topic per section.
- Write out conditions fully: who it applies to, from when, with which exceptions.
Mind the formats
Plain text and well-structured web pages work best. Information locked in scanned PDFs, images of tables, slide decks and screenshots is hard to extract reliably. Where important answers exist only in those formats, convert them to text. Tables deserve attention: a price table that is clear to the eye can turn into a confusing run of numbers once extracted, so check how yours come out.
Decide what the bot must not answer
Some topics should go to a person regardless of what the documents say: legal questions, medical questions, individual account disputes. List them. It is easier to exclude them deliberately at the start than to discover them through an incident.
Give the content an owner
Documentation changes. Prices change, features change, policies change. Someone needs to be responsible for updating the knowledge base when that happens, and the chatbot's index has to be refreshed when content is updated. Without an owner, quality declines quietly over a few months and nobody can say exactly when it started.
Summary
Begin with real questions, close the gaps, remove contradictions, write self-contained sections and name an owner. We run this content review at the start of every AI chatbot project, because it decides the result more than the model does. If you would like help assessing your knowledge base, get in touch.