AI’s Impact on Data Silos and Knowledge Hubs

The overall effect of AI on knowledge hubs and data silos is as multifaceted as the individual technologies shaping this discipline. The gamut of AI techniques that are readily employed for knowledge sharing includes keyword search, vector similarity search, language models, less advanced forms of machine learning (ML), dynamic agents empowered by language models, and knowledge graphs.

These approaches span AI’s statistical and nonstatistical sides. As such, it’s a mistake to consider AI solely as the large language models (LLMs) that have become ubiquitous across the enterprise—particularly for applications of knowledge sharing. In many instances, the synthesis of AI techniques creates or reinforces knowledge hubs and decreases the number and severity of data silos.

However, it’s chimerical to assume that recent advancements in these technologies will eradicate data silos, even with embeddings, agents, and language models.

“The vector approach is great for providing knowledge hubs, but in terms of eliminating data silos, it’s not a silver bullet,” cautioned Stephen Bixby, Pega SVP of product engineering. “That’s not where you want to have your system of record. It’s more like an index of all of that aggregated knowledge put into a searchable place.”

AI impacts data silos primarily by assisting with the consolidation of enterprise knowledge into locales that are uniformly accessible across business units. In that respect, AI techniques are instrumental.

“AI makes it easier to get the information in the first place, which you can then use to either orchestrate processes that themselves help with data silos, or to tie in with conversational AI to get insights out of information or data,” commented Justin Pava, Laserfiche chief product evangelist.

Knowledge Graph Aggregation

There are few approaches more utilitarian for consolidating enterprise knowledge than knowledge graphs. Semantic knowledge graphs exemplify nonstatistical AI with their symbolic reasoning, axioms, and rules-based capabilities. Moreover, they harmonize data of all variations with standardized ontologies and taxonomies, which are ideal for assembling knowledge across silos.

However, these traditional advantages may seem tame compared to contemporary technological developments, which enable users to facilitate natural language interactions with LLMs to “list all the repositories you have,” explained Franz CEO Jans Aasman. Top knowledge graph approaches supplemented with language models can extract the ontology and schema of each repository organizations have, allowing users to query across them.

This functionality applies to non-graph databases, including relational ones and other forms of NoSQL databases. Aasman described a tool that “can take the schema of a PostgreSQL database, or the shape of a MongoDB database, or Parquet database, and turn that into an ontology.” By storing the underlying schemas from these databases in a metadata knowledge graph, organizations can query each of these databases—or all of them with one question—via a language model that interfaces with an enterprise ontology at the top of the stack. Consequently, these databases are uniformly linked and accessible, so they’re no longer siloed.

Query Sophistication

Bixby’s vector approach comments highlighted the widespread use case for employing AI to aggregate enterprise knowledge and make it searchable. The method Aasman articulated excels at both. In fact, it delivers a degree of query sophistication that may not be possible without this approach.