Knowledge base architecture is the real RAG problem

The model is rarely the bottleneck

When a retrieval-augmented system gives a wrong answer, the instinct is to change the model. In our experience the cause is almost always upstream: content that is missing, duplicated, conflicting, ambiguous, out of date, or structured in a way that makes the right passage unretrievable.

A better model applied to that corpus produces a more fluent wrong answer.

Four failure modes worth auditing first

Duplication. Four articles describe the same procedure, three are stale, and retrieval has no way to know which is authoritative.

Hedging. Content written to avoid commitment ("generally," "in most cases," "contact your administrator") gives the model nothing to answer with.

Wrong granularity. A forty-page policy document chunked into fragments that each lack the context needed to be useful alone.

Missing scope. Articles that never state who they apply to, so the system cheerfully applies a contractor policy to a full-time employee.

What good architecture looks like

A taxonomy that reflects how people ask, not how the organization is structured. One authoritative article per procedure, with a named owner and a review date. Explicit scope and audience metadata on every document. And chunking decided by the shape of the content, not by a default character count.

Governance is the part that lasts

Cleansing a knowledge base is a project. Keeping it clean is an operating model: who may publish, who reviews, what triggers a re-review, and what happens to an article whose owner leaves. Without that, the corpus degrades back to its original state in about eighteen months.