Predictive analytics: forecasting future trends from existing data

By John Harney

With the growth in customer relationship management (CRM) and business intelligence (BI) tools, companies have found it relatively easy to determine which customers are purchasing most of their products. Generally, 80% of the product sold is purchased by the company’s top 20% of customers. But what if a company wants to predict which customers will purchase the most product over their lifetimes? BI can’t help in that situation. That’s where a new discipline called predictive analytics comes in.

Predictive analytics is a subset of data mining that “enables you to derive new insight or new information from existing information,” says Walter Janowski, research director of Gartner's CRM Practice. By contrast, he says, BI tools take past information and let you churn it and present it in different ways. For instance, you can break out customer groups by gender and age and figure out which groups are buying which products. Predictive analytics, on the other hand, says Janowski, lets you perform activities like predicting “estimated lifetime customer profitability—going beyond how much a customer has spent so far to how much we anticipate that customer to spend over his lifetime with us as the seller.”

Uses of predictive analytics

Definitions and discussions of predictive analytics tend to remain general unless they are tied to specific examples. For instance, a company might use predictive analytics to upsell certain customers. Laurie Orlov, research director, Business Applications, Forrester Research, says that predictive analytics can determine a candidate for additional services. Imagine, she explains, that “a person has this history of buying, an account of this size, and the following demographic, which means that they probably have an income of xyz ... ” Given those conditions, they are ripe to buy. Therefore, she says, “We offer them x.”

Predictive analytics is also commonly used to determine customer churn rates. If a customer shares certain characteristics with other customers who have defected to other vendors, it’s likely that customer will leave also. To retain such a customer, the vendor could go out of its way to stroke that customer with special offers or attention.

Similarly, it could also be used to determine the best sales channel(s) to use to reach customers, says Bob Blumstein, research director of CRM Analytics and Marketing Applications at IDC IDC. He explains that one customer might usually interact with the company by visiting its Web site while another might better respond to the company’s direct mail. Blumstein explains, “You want to know which channel is the best channel, which combination of channels is best, how much communication in each of those channels is optimal to the relationships, at what point are you wasting your dollars or … alienating your consumers?” Such models are not merely useful on the sell side, though. Similar ones could be used to predict supply chain demand.

Such applications tend to focus around CRM, but fraud detection applications are just as common. For instance, many packages can be used to track certain factors that define a credit card user’s buying behavior. Bob Moran, research VP and managing director, Data Knowledge and Analytics, Aberdeen Group, says, “ If you have somebody who is displaying a specific type of usage compared to the type of usage one is conditioned to with a specific credit card, you can predict whether that usage is fraudulent or bona fide.” For example, if the owner of the card usually travels in known regions of the world, but card usage starts appearing in other geographical regions, that spending pattern could indicate someone other than its owner is using that card.

Other applications are valuable also, though admittedly some are a bit arcane. One vendor even takes the weather into account when determining how to distribute products.Planalytics keeps a history of weather patterns over the last 30 years, according to Orlov. "Firms can look at their own sales and look at that data and decide how much to stock of a particular item (say, Duraflame logs) in a particular part of the country based on the possibility of an unusual cold spell," she says.

Henry Morris, VP for Applications and Information Access at IDC, says that predictive analytics is also a great tool for doing quality analysis in the manufacturing process—for instance, predicting when a piece of equipment will fail given the factors that existed when similar equipment failed in the past.

Orlov goes so far as to suggest that predictive analytics could be a powerful tool in fighting terrorism. She says that authorities with the right tools can monitor data banks for information like a suspicious person’s visa status and firearm registration, and then extrapolate from that data to see if the individual in question fits a common terrorist’s behavior profile. For example, if airport security saw a suspicious person frequenting an airline terminal, they could detain him, check his background and if the findings warrant, do more in-depth research on things like his travel activity. The results, when compared to other terrorists’ profiles, might reveal his link to a terrorist group.

User benefits

According to Orlov, “Any company that uses predictive analytics effectively can have a better handle on future possibility ... and that would include future sales possibilities, future costs to them, future possible risks.” For instance, imagine a store understocks air conditioners because it has no way to correlate heat waves to its supply of air conditioners. If it had used predictive analytics that factored in historical weather patterns, she says, it might avoid understocking as well as derive such related benefits as not having to pay to ship items from one part of the country to another.

Janowski points out that companies using predictive analytics can identify who their most profitable customers are. “Then they can deliver differentiated levels of service. They can focus their resources on that 20% of customers that is generating 80% of the revenue,” he explains.

As those examples illustrate, most benefits are tied directly to the application of the tools; detecting credit card fraud, for example, would be the benefit of using predictive analytics for that purpose. But Moran sums up the across-the-board benefit as “risk minimization.” Knowing what to expect and determining how you will deal with it reduces its possibly deleterious effect. On the other hand, reducing risk also optimizes your chance of success--whether it’s knowing who your best customers are or knowing how much you can charge for a product to improve your margins without overpricing it and alienating buyers.

Tips for users

Despite the numerous benefits, Janowski warns that users should not look at the package as a "magic box" to solve their problems. Rather, he suggests, “Approach it from a goal-based direction—in other words, what do you want to know? If I know this about my customers, I could do this type of action. Then you can find the data you need to get there." He adds that you also can answer critical questions like “do you already have that data, do you need to collect it, what exact tools do you need to analyze it?”

Orlov cautions, “You need function-specific people for function-specific analytics.” If you’re using predictive analytics to focus a marketing campaign, she says, don’t isolate the technical users of the tools from the marketing experts who will benefit. Their input is crucial. As with other BI tools, predictive analytics is only effective if it determines decisions that front-line executives make about key corporate activities. More specifically, predictive analytics tools should be easy to use and usable for different types of applications. Moran, for instance, suggests that users “be concerned about speed and scalability—in other words, if a data mining tool is not built for scalability, it will not be able to churn through the data in a sufficient amount of time to make the results extremely useful." He also recommends that users determine if there is a framework in which they can apply those technologies over and over again in slightly different ways. “For instance,” he says, “if I do fraud detection, is there a way I could use the same backbone to do customer relationship management?”

To improve efficiency, look for a mechanism that allows for a large degree of repeatability. Moran explains, “Can you iterate through your data and make slight changes to the data set rather than having to do a one fell swoop change over and over again, which is time-consuming and expensive? I decide that I don’t have the right data, what do I do? Do I start all over again or update the data set with another increment of data? Some tools make you start all over; others allow you to iterate using a slightly different data set.”

Users might even look for prepackaged tools. Blumstein says that some tools are “not just a raw regression algorithm, but rather … might be a packaged regression algorithm that runs in a semi-automatic sense and puts out a series of reports that are keyed exactly to the functionality that the business user is going to apply it to.”

Observes Orlov, “Any of these tools are only as good as the data they have to canvass. So some of the gotchas are inadequate data, not recent data, or inaccessible data.”

Statistical forecasting has been around for years. Predictive analytics is a leap beyond traditional methods because it offers faster processing speeds and more complex algorithms that also operate on top of existing data mining infrastructures. That means they can analyze more data for less of an investment. As a bonus, the tools themselves are also much easier to use than ever before. What’s not to like about that?

.....................................................................................................................................................................................

John Harney is president of ASPWatch, a consultancy that delivers strategy for application service providers, e-mail jharney897@aol.com.