Building a Company Knowledge Base an AI Assistant Can Use

Most organizations already have a knowledge base. It’s just scattered across a shared drive, a few people’s inboxes, an old wiki nobody updates, and the memory of whoever has been there longest. A person can work around that by asking around. An AI assistant can’t. It works from what you give it, and if what you give it is a folder of forty files with names like “FINAL_v3_use this one,” it will pick the wrong one with total confidence.

Building a knowledge base for AI is mostly about organizing what you already have so the right information is easy to find and the wrong information is out of the way. I built a client knowledge base template for this: a starter folder structure, an instruction file that tells the assistant how to use it, and an onboarding ramp so a team can fill it in over a few weeks instead of all at once.

What goes in

Start with the information people ask about most and get wrong most often. For most businesses, that’s a short list.

• Products and services, with current specs and descriptions
• Pricing rules and who can approve exceptions
• Common customer questions and the answers you actually give
• Procedures for recurring work: onboarding a customer, handling a complaint, closing out a job
• Examples of good work: a strong proposal, a clear customer email, a report someone praised
• Policies people need day to day: returns, warranties, safety rules

What doesn’t belong: drafts, personal notes, old versions kept “just in case,” and anything sensitive that the assistant has no reason to see. If a file isn’t true anymore, archive it outside the knowledge base. Leaving it in is how an assistant ends up quoting last year’s prices.

A structure an assistant can follow

Simple beats clever. A few top-level folders with obvious names will serve you better than a deep tree nobody can navigate.

The instruction file at the top tells the assistant what each folder holds and which file wins when two disagree. That one file does a lot of work. Without it, the assistant treats a two-year-old example and the current price list as equally reliable.

Naming files

File names are the first thing an assistant reads, so make them describe what’s inside. “warranty-policy.md” beats “Policy Doc 2.” Use the same pattern everywhere: lowercase, hyphens, no dates in the name unless the date is the point. Put the date and owner inside the file instead.

I start each reference file with a few lines like this:

Owner: Jen Alvarez (operations)
Last reviewed: [date]
Status: current
Scope: Warranty terms for all residential installations. Commercial terms are in commercial-warranty.md.

That header tells a person and an assistant the same things: who to ask, whether it’s still trusted, and what it doesn’t cover.

One source of truth

Every fact should live in one place. If your price list is in the pricing file, the sales playbook, and three proposal templates, it will be wrong in at least one of them by next quarter. Keep the fact in one file and have everything else point to it.

This takes discipline at first, because copying is easier than linking. But it’s the difference between a knowledge base that stays accurate and one that slowly fills with contradictions.

Permissions and sensitive information

Decide early what the assistant should never see, and keep it out of the knowledge base entirely. Payroll, personnel files, customer financial details, anything covered by a confidentiality agreement. Separate folders with different access are fine for people, but the simplest protection for AI work is not to put the material where the assistant can reach it.

If different teams need different information, give them separate knowledge bases rather than one big one with rules about who can read what. Rules get forgotten. Folders don’t.

Keeping it current

A knowledge base starts going stale the day you finish it. Three habits keep it alive.

• Every file has an owner, and the owner’s name is in the file.
• Every file has a review date, and someone checks the dates monthly.
• When the assistant gets something wrong because a file was wrong, fix the file that day.

That last one matters most. Each mistake is a free audit. Fix the source, and the same mistake doesn’t happen again.

An onboarding ramp

Don’t try to build all of this at once. The template I use spreads it over a few weeks.

1. Week one: write the instruction file and add the five reference files people ask about most.
2. Week two: add your three most common procedures and one good example of each.
3. Week three: add templates for the documents you produce most often.
4. After that: add files as gaps show up in real work.

By the end of a month, the assistant has enough to be genuinely useful, and the team has built the habit of keeping it current. That habit matters more than the files. A knowledge base that’s eighty percent complete and accurate beats one that’s complete and half wrong. And a person still checks the work before it leaves the building.