Best Practices in Cognitive Computing

What is Cognitive Computing?

We are still at an early stage of an evolving market, where many players try to position their offering as “cognitive.” The Cognitive Computing Consortium is trying to come to a consensual definition, such that potential user organizations get a more objective idea of what to expect from cognitive computing. Here you can find their current definition. It is a bit long so let’s try a shorter one:

A cognitive information system is capable of extracting relevant information from big and diverse datasets for users in their work context.

As a corollary, such a system must be able to “understand” people’s needs and their work context. Needs can be expressed as questions in natural language or by users subscribing to certain topics, asking to be alerted if anything relevant happens. Information needs can also be implicit: the Cognitive Computing system knowing the work process of users deduces at which points they will most probably need certain information to do their work properly.

“Understanding” and “knowing” are, of course, not used here in the human sense (nor in the sense of “strong AI”). They rather mean that a combination of natural language processing (NLP), statistical analysis and machine learning algorithms can take a good shot at providing relevant information for users at opportune moments in time.

With the definition of cognitive computing still being in flux, it is over-optimistic to expect “best practices”—and under-ambitious to just want to copy what others did. We hope use cases inspire people to come up with the “best practices” for their organization, rather than using them as ready-made blueprints without adaptation.

Successful use cases have several things in common:

A Large Set of Connectors to Diverse, Internal and External Data Sources

It takes a lot of grapes to distill a single bottle of brandy. And it takes a lot of diverse data to distill relevant information. You need to get at that data rapidly if you don’t want to be bogged down in low-level IT “plumbing.”

Off-the-shelf connectors speed up your projects by an order of magnitude.

Structured and Unstructured Data

Around 80 percent of enterprise data (internal and external) is unstructured, meaning it is not numerical data from relational data bases, produced by enterprise applications like ERP, CRM, MDM, PLM, etc. It is to a large extent textual data from reports, publications, contracts, letters, email, but increasingly also from images and video.

You need to deal with all of this data in order to extract relevant and valuable information.

NLP and “Classical Analytics”

NLP is needed in two distinct cases: to “understand” the questions of users expressed in their native language, and to “understand” the content of texts in different languages. Text analytics delivers semantic understanding, sentiments expressed in a text, and relationships between concepts. Concepts may be names of people, places, companies, products, components, molecules, but also license plates, phone numbers, email addresses, bank account numbers, etc.

Combining the analysis of structured and unstructured data delivers better results much faster than dealing with each data category separately. For example, if you can look up a person’s name in the company LDAP directory or a product name and its synonym (e.g., scientific name of a drug), you instantly know much more than you could by general concept extraction and correlation.

A Logical Data Warehouse as a Unified Basis for Information Applications

The results of plowing through the vast and diverse sets of data form a reservoir of information and knowledge that we call a logical data warehouse (LDW). It is continuously enriched by the results of analysis (semantics, statistics, and machine learning as we will see below), and forms an ideal basis for “information applications” (InfoApps) that cater for specific business needs.

The LDW shields developers of InfoApps from the complexities of the original data sources and offers unified data access. They can thus be much faster and more efficient.

The speed of app development offered by a good LDW is groundbreaking for most IT teams. But it is this level of innovation speed that is necessary to stay agile and proactive in industries where speed has become essential.

Machine Learning on the Enriched LDW

Machine learning (ML) on raw data requires more and better data than most enterprises have at their disposal. And it takes potentially very large numbers of iterations for ML algorithms to produce actionable insight.

Turning ML algorithms into the enriched LDW, where concepts have been extracted and perhaps even related to each other (by company or industry dictionaries and ontologies), allows ML to work on a richer data set which is then enriched again by the results of the algorithms.

Use Case: Find Networks of Experts

Looking for experts on a given subject is an important use case in many industries. You need experts to cover different aspects of a piece of technology: from material science via electronics to programming. In the biopharma industry you need experts in medicine, pharma, biology, genetics, etc.

Experience shows that networks of experts cannot be found by looking at CV collections or the postings of enterprise social networks. You need to look at the “footprint” they leave in writings and data, including publications, product descriptions, project reports, and even emails and postings in social media. That is why you need a cognitive computing system to find the right experts for a new project.

ROI can come in various categories, such as rapidly creating a team of experts for drug repositioning where speed is decisive. Detecting redundant or overlapping R&D projects and connecting experts amongst each other can save hundreds of thousands if not millions of dollars.

Use Case: Create 360° Views

Enterprises need 360°-views on many kinds of subjects: products, supply chains, customers, competitors, etc. Let’s focus on customers here.

In most companies, customer data is spread across dozens and sometimes hundreds of applications. Integrating these systems means creating a single data model—a daunting task in most companies.

Using cognitive search and analytics to build an LDW rather than a classical data warehouse is much more flexible, and projects are usually an order of magnitude faster.

ROI of a 360°-view of customers for a call center with 10.000 agents was in the region of $60 million over 3 years.