Assemble trustworthy AI with KM and RAG

RAG (Retrieval-Augmented Generation) has gained much attention lately with its promise to reduce, or even eliminate, hallucinated information from search results.

By combining language models with contextually relevant, organizationally verified knowledge stores, retrieved data is transparent, accurate, and reliable, with underlying sources identifiable.

KMWorld recently held a webinar, KM + RAG: Building Trustworthy, Context-Aware AI, with Graphwise and Shelf about how ontologies and structured knowledge create richer context for superior AI and more.

According to Kent Stroker, pre-sales engineer at Graphwise, marrying KM with RAG grounds enterprise language models in verifiable semantic knowledge graphs, to reduce hallucinations and prove every answer explainable and audit-ready at scale.

Taxonomies and ontologies as living assets, owned by named people, federated across systems rather than centralized into one model nobody can agree on, he noted.

Every operational system speaks its own local dialect. A single top-down enterprise model stalls in committee, so model each source on its own terms and bind them by mapping.

To successfully integrate and trace lineage, Stroker recommended the following:

Leave the data where it lives: Copying data into a central graph buys synchronization lag and a second governance problem. Federate through a virtualized semantic layer over the existing stores. Query databases, lakehouses, and APIs in place, without a migration project. The system of record stays the system of record and stays authoritative.

Surface the lineage, do not bury it: Trustworthy AI means every generated answer can show its working to the person who must act on it. Capture standards-based provenance across queries and transformations. Keep timestamped trails of the exact source records and schema versions. Put traceability in front of the reader, not in a log they will never open.

Semantic Tagging of Documents should be:

• Automated entity extraction and resolution: Policies, contracts and correspondence run through ontology-guided extraction that maps each textual mention to a known identifier, rather than leaving it as a string.
• Connecting unstructured text to structured truth: Once tagged, a contract is joined to the vendor, the transactions and the organizational hierarchy already modelled in the graph.
• Context expansion through the neighborhood: Retrieval returns the surrounding subgraph as a function of the prompt itself: parent definitions, upstream dependencies and the obligations that attach to them.
• Disambiguation and polysemy: The ontology settles which sense is meant, separating a product codename from the department abbreviation it collides with, before the model ever sees the text.

Jan Štihec, director, data and AI at Shelf, said RAG promises answers grounded in verified, transparent, identifiable sources. Getting there at enterprise scale takes more than chunks and embeddings.

Messy enterprise knowledge can, to an extent, be worked around by people, but it does not scale into reliable AI, Štihec explained. A truth layer makes it reliable and usable. Companies should create one reliable foundation of governed, contextualized knowledge that every AI application can use, instead of a separate cleanup for each one.

Ontologies let you build a digital twin of a process. The twin represents both the data and the business logic that governs it, and AI needs to understand both, Štihec said.

Enterprise knowledge is too large and changes too quickly to structure and govern by hand. The truth layer must build and maintain automatically.

For the full webinar, featuring a more in-depth discussion, Q&A, and more, you can view an archived version of the webinar here.