Putting intelligent search to work

Taxonomy automation Symbolic reasoning employs a number of semantic technologies and approaches to make intelligent inferences about business-defined concepts. Including it with additional AI methods involved in cognitive search creates the powerful advantage of building taxonomies without the exhaustive efforts of doing so manually. “You’re trying to automate that as much as possible on behalf of the users as opposed to turning that into a massive modeling exercise,” Potter said. “What you’re really doing is defining many elements of an expert system, and you’re trying to do that dynamically.”

The hierarchical comprehension of the meaning of terms is crucial to the underlying knowledge for advanced reasoning. The options in this space can automate this process by obtaining “all kinds of information at the knowledge extraction level: terms, specific entities, snippets of text, combination terms,” Welsh said. “Those are related all together in a representation that measures the distance and relatedness between those terms in a multiple dimensional graph structure.”

This hierarchy of knowledge then becomes the foundation for rules-driven approaches to understanding natural language. “The reason for rules-based approaches is accuracy is really important for businesses,” Burnett remarked. “If you’re going to make business decisions off the back of what you’re being told, accuracy is important. I’ve used machine learning language models for complete end-to-end language generation, and they see things that aren’t there because of the data they’re trained on.”

Learning models

Burnett’s perspective is important for two reasons. First, it reinforces the ongoing value of taxonomies as providing infallible “accuracy” in the age of machine learning. Second, it brings up the principal warning about relying solely on machine learning for cognitive search or natural language applications—its reliance on training data. This concern is typified by supervised learning use cases in which labeled training data is often scarce, expensive to find and annotate, and inordinately time-consuming to do.

Organizations can sidestep this issue that frequently derails machine learning projects with techniques relying on unsupervised learning, which substantially decrease the time to value for cognitive search solutions. According to Pengali, “The advantage of unsupervised learning is all of that labeling is not required. So, what you’re finding, at some intuitive level, is similarity between things as opposed to some label.” Top cognitive search systems, therefore, utilize both of these machine learning techniques to accelerate the time it takes to begin using them for conversational interactions with data sources.

In most instances, pairing them is instrumental for constructing the knowledge or taxonomic base for implementing the rules-based accuracy Burnett described. For example, if a company wanted to search through a discrete series of documents for a specific use case, these statistical measures would “machine learn knowledge from the underlying documents,” Welsh said. “It understands things you may not have explicitly seen before, and that gives you a usable system out of the box.” The speed of this functionality is advantageous, particularly when considering the reams of training data organizations are otherwise required to produce and pay others to label, or the time and effort necessary to manually build taxonomies.

Takeaways

The key takeaways from contemporary enterprise search techniques are multifaceted. As a discipline, search is not only firmly entrenched within the realm of AI—and always has been—but also, as a more modern consequence, it has become intrinsically conversational in nature. This expedites the issuance of results, improves their quality, and exponentially boosts the underlying value generated from enterprise search.

Another key takeaway is that search operates as practically the nexus point of AI in the truest sense of the term. As previously noted, cogent developments in this space draw from both statistical machine-learning approaches and non-statistical knowledgebase approaches. “It’s a mix of symbolic and machine learning,” said Aasman. “You can call it neuro-symbolic.”