Big Data = Big Value: Outwit Your Competition

"Big data" is receiving big hype these days, but when it comes to taking action, many unfortunately are taking a wait-and-see approach. Big data can be intimidating, but like learning any new skill or sport, it's perfectly acceptable (and recommended) to start small. If embarking on a big data project is like completing your first marathon, you wouldn't compete without first training and building up endurance. Similarly, with big data, I recommend starting smaller and smarter. Remember, big data is not always just about size (volume); variety and velocity are equally important.

This major shift in thinking illuminates the need to integrate data from multiple sources (variety), and to capture that data in real time (velocity). It demands that we broaden our perspective beyond in-house data and consider how to access the ultimate source of big data—the Internet itself—and capture all the data needed for predictive analytics.

But here's where many people grind to a halt, because they're intimidated by the volume of data they think they need. Or perhaps they don't know how to access the data they want because it's not neatly stored in a relational database or data warehouse.

These people should take some lessons from the book "Moneyball." If you're not familiar with "Moneyball," it's the fascinating story of how the Oakland Athletics baseball organization built a championship team by taking a smarter approach to using and analyzing data as a competitive advantage.

The Right Data

How did Oakland's management predict which players would perform the best? They started with the same historical information available to the other teams, but instead of relying on gut feel or traditional, non-data driven approaches, they built their team on objective measures that best predicted a higher winning percentage.

Selecting the data you need to develop your own company's unique insights can be a challenge, but it's likely that your organization can identify several indicators that can have a strong influence on future performance. Don't let your search for raw data be limited to your in-house relational databases. It's now possible to access and integrate data from virtually anywhere on the Internet, without scripting languages. So just focus on the data that provides the most value, even if it's outside your firewall.

Relevance and Accessibility

As you begin planning your own "Moneyball approach to big data" you'll encounter some challenges the Oakland Athletics didn't have: coping with the Internet's wide variety of data, and staying up-to-date with the fire hose of data it can produce. But these factors don't need to change your basic approach. For example, Twitter handles some 250 million tweets daily, but when only 1,000 of them relate to your product or company, there's no need to retain all 8 terabytes of tweets sent each and every day.

Of course, Twitter isn't the only source of data that may be relevant to your organization. Blogs, review sites and news sources represent one type of potentially relevant data, but Web-based applications offered by your partners and competitors are another source that can add relevance to any analytical effort. You will need the ability to access a wide variety of data, and to access it in real time—because the quality of the conclusions you can draw from your data is much less dependent on the volume of data than it is on the relevance of the data and your ability to access it.

Big Data: It's for You

Any organization can benefit from Internet-based big data: a third of executives surveyed by IBM have made decisions with inaccurate or untrusted data and half of them believe they don't have sufficient information from across their organization. Decision-makers are demanding more and better information, and when Forrester estimates that enterprises currently use less than 5% of available data it should be clear that you don't need to reach for 100% all at once. A small increase in the amount of data you capture—from 5% to 6%, perhaps—can still offer tremendous benefits.

Be aware that a real-time application-integration platform will enable you to create more personalized products that delight your customers, automate real-time feedback on your brand and your products, and produce your own custom research that allows you to see trends before your competitors do. You will add to the top line, improve operating margins and make your organization much more agile—so you can stay ahead of the competition.

Don't let the size of big data paralyze you; get real-time access to the data that is relevant to your company's growth—and take action today. 

Case Study: Online Retailing

One customer, an online sporting goods retailer, developed profitable insights by integrating data from both sides of their firewall. By analyzing hundreds of thousands of sales transactions, they learned that a select few items influenced a large portion of their revenue. When a new customer bought a fishing rod, for example, they were more likely to buy other angling gear, generating a high CLV (customer lifetime value) in the process.

They initially attempted to manually monitor the websites of their major competitors to better analyze the impact of these critical revenue generators, but they found it impractical even at low volumes—and the 24/7 nature of online retailing demanded real-time responses. Instead of continuing manually, they chose to automate with Kapow Katalyst. The initial implementation took a few days, and enabled them to detect and respond to special pricing and promotions on their chosen revenue influencers.

Kapow Katalyst automation has enabled several "Moneyball moments" for this customer. In one instance, they learned that certain discounts were ineffective for attracting and retaining high-value customers, so they were phased out. In another they found they were able to detect merchandising patterns at certain large competitors, and develop pre-emptive campaigns—and offer some of the phased-out discounts where they could be more effective. They are even able to study the competition and gain insight before launching product line expansions.

This smarter approach enabled them to build a more intelligent enterprise. They can analyze thousands of products across hundreds of competitor websites, and integrate this insight into their real-time action-oriented decision-making.

The Kapow Katalyst Application Integration Platform gives you unprecedented agility in delivering IT solutions to urgent business demands for big data, application integration, cloud integration, process automation, content migration, mobile-enablement, business intelligence and Web intelligence. Visit kapowsoftware.com or call 800-805-0828 for additional information.