Cognitive Search Brings the Power of AI to Enterprise Search
What is Cognitive Search?
Forrester, one of the leading analyst firms, defines Cognitive Search in a recent report¹ as: The new generation of enterprise search that employs AI technologies such as natural language processing and machine learning to ingest, understand, organize, and query digital content from multiple data sources.
Here is a shorter version, easy to memorize: Cognitive Search = Search + NLP + AI/ML
Of course, “search” in this equation is not the old keyword search but high performance search integrating different kinds of analytics. Natural language processing (NLP) is not just statistical treatment of languages but comprises deep linguistic and semantic analysis. And, artificial intelligence (AI) is not just “sprinkled” on an old search frameworks but part of an integrated, scalable, end-to-end architecture.
AI Needs Data, Lots of Data
For AI and machine learning (ML) algorithms to work well, they need to be fed with as much data you can get at. A Cognitive Search platform must access the vast majority of data sources of an enterprise: internal and external data of all types, data on premises and in the cloud. Hence, the system must be highly scalable.
Continuous Enrichment
Cognitive Search accumulates knowledge about structured and unstructured data and about user preferences and behavior. That is how users get ever more relevant information in their work context.
To accumulate knowledge, a Cognitive Search platform needs a repository for this knowledge. We call that a “Logical Data Warehouse” (LDW). The LDW contains information about data, it is continuously enriched by NLP and by the results of clustering and similarity calculations, etc. Hence, ML algorithms and any other “intelligent analytics” must work on this LDW and feed their results back into it.
The Strength of Combination
To produce the best possible results, the different analytical methods must be combined, not just executed in isolation of each other. For example, unleashing ML algorithms onto textual data for which linguistic and semantic analyses have already extracted concepts and relationships between concepts, leads to much better results much faster.
Security/Access Rights
A Cognitive Search platform must not allow individual users to circumvent access control. Even at the end of an elaborate round of ML or other enrichment processes, no one must be able to see information they should not be allowed to see.
Use Cases
360° views and finding networks of experts continue to be the two most widespread use cases. In fact, many use cases can be presented as 360° views on a particular subject, be that “subject” the customer, a product, a research subject, etc.
Let us consider, nevertheless, two specific use cases in more detail.
Use Case: Regulatory Compliance
Here, companies scan the output of regulatory agencies across the world (in all major languages), find out what the new regulations are about, and whether they concern one of their products. If so, they push the relevant information on these new regulations to the relevant people within the company and its suppliers.
Recognizing what is relevant requires deep NLP processing—and ML with training sets where people have categorized regulations “by hand,” sometimes over years, without being able to define a set of rules for doing so.
A related facet of this use case is “listening to the voice of the customer”: what do people say on social media about your products. For pharma companies, that includes detecting new side effects being discussed by patients. In this case, speed is of the essence: Regulators often impose strict time limits for reporting safety-critical or health related effects detected by listening to the voice of the customer.
Benefits range from fast reactions to negative posts via the detection of flaws and their rapid correction to lives saved in health and safety related cases.
Use Case: Maintenance and Repair of Complex Systems
Consider complex systems such as aircraft or helicopters, trains, power stations, etc., with enormous numbers of components, often in unique combinations, making such a system one of a kind.
Problem reports on such a system can be human-generated or come from sensor data or both. In each case, a human operator has to be guided towards doing the right thing at the necessary speed.
This involves plowing through voluminous and complex product documentation, maintenance/repair procedures, regulations, reports, and detecting relevant information. And, it requires heeding interdependencies of various components. Reports on similar cases with successful solutions may greatly speed up the process and provide precious help to operators.
Users must be made aware, for example, that they cannot change one piece without changing others; that certain pieces cannot be used in certain configurations, and which pieces are interchangeable with others; which tests are obligatory after certain replacements/repairs. They must realize when a malfunction has to be reported or even alerts to be issued to owners of systems with similar configurations.
In addition, a Cognitive Search and analytics system is expected to keep track of incidents, discover related ones, and compute trends.
The sheer volume and interdependence of information to be extracted from data requires a Cognitive Search and analytics system assisting human operators. It is as easy to have a vague idea of where NLP and ML will help, as it is difficult to explain it in detail. Imagine!
When the safety of people is at stake, ROI calculations may be inappropriate. To keep planes in the air, trains on track, and power stations safe, the right information must be found and rapidly made available to the right people. Nevertheless, Cognitive Search and analytics can yield significant ROI by amplifying what teams can do without amplifying their size.
1The Forrester Wave™: Cognitive Search And Knowledge Discovery Solutions, Q2 2017, downloadable from the Sinequa website