-->

KMWorld 2024 Is Nov. 18-21 in Washington, DC. Register now for $100 off!

Overcoming data silos: A range of options

Article Featured Image

Searching across the enterprise

Search technology has been a mainstay of KM for many years. Its role in bringing together virtualized data so that the source is transparent to the user makes it one of the most efficient ways to over-ride the limitations of disparate repositories. “SearchBlox’s SearchAI can find and summarize information across silos,” commented John Lewis, CKO of SearchBlox. “It uses a private LLM [large language model] to generate responses but offers more control and direct oversight by using local content for retrieval augmented generation (RAG).”

SearchBlox produces a line of enterprise search products and specialized AI-enabled products, including SearchAI ChatBot and SmartFAQs, that provide natural language answers to users’ questions. The latter product automates the generation of FAQs by searching across data silos and providing answers to common questions through generative AI. “This year has been historic in terms of the use of generative AI in search,” Lewis noted. “The field is moving quickly, and users’ expectations are increasing. Companies need to offer AI-driven chatbot conversations that sound authentic and search results based on the best information in the enterprise, wherever it is stored.”

The steadily increasing availability of connectors has paved the way for easier integration with a wide array of data sources. Nearly 10 years ago, Lucidworks Fusion v1.0, an enterprise search solution, shipped with 25 connectors; it now offers 800, including integration with cloud services (Google Cloud Platform, AWS, Microsoft Azure), databases (Oracle, DB2, MySQL), and ecommerce platforms (Shopify, Adobe Experience Manager). Content management solutions supported include Alfresco and Box, along with collaboration products such as Jive and Slack. Other platform categories supported include ERPs, CRMs, and MDMs.

Lucidworks is built on the open source search tool Apache Solr and adds numerous enterprise-grade features, including text analytics and machine learning. “Lucidworks provides ETL, and integrates with systems of record at query time and index time,” said Max Bell, senior systems engineer at Lucidworks. “We also use generative AI to summarize documents, which is more useful than just showing the first few lines of the text.” The ability to search across disparate repositories removes the obstacles that arise when data owners have to coordinate to centralize the data. “Each department has its own data,” Bell noted, “and it can be difficult to get everyone to agree on how to bring it together.” Lucidworks creates a single index that spans all the selected repositories, making searching comprehensive and efficient.

Breaking down document silos

In a document-centric work environment, being able to search and manage information across multiple repositories has a direct impact on productivity. “Many document management systems come into organizations as silos and may not be visible or accessible to workers who could potentially benefit from them,” said Antti Nivala, founder and CEO of M-Files. “Our platform is repository-neutral and can integrate with anything from network folders to Microsoft 365 to Salesforce, SharePoint, and SAP, among others.”

The information does not need to be migrated into M-Files, although it can be. While remaining in its original repository, the information can be enriched with metadata, workflow, and versioning. “The changes can be set up to be unidirectional or bidirectional, depending on whether the organization wants the changes to only be maintained in M-Files or reflected back to the original documents,” Nivala added. M-Files also provides automatic classification and entity extraction.

In addition, M-Files can crawl various repositories and identify the title, author, the emerging generative AI technology, such as interacting with systems faster by asking rather than clicking, keeping data safer and more secure, and sparking creativity. “Many repetitive tasks are not yet automated that could be,” he suggested, adding that AI is not always required for automation. A McKinsey report discussing which tasks could potentially be automated states that 10%–20% of knowledge work tasks could be automated. The most likely tasks for automation would be the more repetitive and predictable ones, which could alleviate pressure on knowledge workers, who would not be reluctant to relinquish those tasks.

KMWorld Covers
Free
for qualified subscribers
Subscribe Now Current Issue Past Issues