Processing Structured Data to Help Smart People Get Smarter
Most organizational theory explores how organizations function. But little is known about the vast amounts of data that an organization generates and how it is processed into knowledge to significantly increase intelligence and achieve better success.
Further complicating the problem is technology that has given organizations the ability to collect and store vast amounts of information into data warehouses. To build a strong business intelligence solution, organizations need to better utilize this data and integrate it into the knowledge-building process.
To better understand the issues, it is important to define what is meant by structured and unstructured data. The unstructured data of an organization includes e-mail correspondence, text documents, even voice and video. But in large part, most of the information in an organization is structured. This is the quantified information found in financial statements, statistical reports and other sources that include responses to surveys, point of sale information and sales reports. In essence, the non-text data the organization generates.
Structured data is extremely difficult to organize. It is generally fragmented, resting in different silos of the organization. Often, organizations have little ability to aggregate the data and process it in a manner that reveals the core information that constitutes a robust knowledge base.
New and proven approaches to processing structured data are quickly emerging.
Integrated Self-learning
CAMO's approach is to integrate a self-learning system that continually aggregates data and correlates it into a knowledge base. When applied to an organization, intelligence increases exponentially as the "community" adds their own content and feeds knowledge back into the system. The system itself is always learning by continually processing structured data from multiple data sources, both internal and external to the organization. We believe that hybrid methods to integrate artificial intelligence and advanced modeling techniques are the most powerful ways an organization can process data into knowledge. What results are sophisticated ways to classify customers, define and track multiple market segments, make predictions, optimize processes and simulate market conditions for product concept development.
The CAMO system gets smarter as new data is aggregated, processed and correlated. Correlating data is an important distinction. It means that the knowledge management system automatically discovers the relationships of all the different structured data sets by creating a model for any possible variable, be it a customer's quantified feedback, the various features of a product or any possible circumstance, state, or condition. This means that the analysis may have tens, hundreds or even thousands of models that interact with each other and make correlations that become the knowledge base. The knowledge base may update itself on a continual basis as new data is aggregated and run through the system's self-learning algorithms.
To be of value, the people in the organization must trust and respect the knowledge base that is generated from the structured data. The keys to value are accessibility, usability, extensibility and robustness. Users must be able to easily gain access to various knowledge bases, to navigate and query with applications that intuitively direct the user to information critical for decision support or knowledge discovery. The knowledge base must also build over time, as new structured data becomes available. This process should extend the knowledge and help people continually learn as the knowledge base builds. This building process should lead to robust knowledge bases with applications that provide user access to a broad range of queries for knowledge discovery or coverage of situations for decision support.
The Process
To develop a knowledge base, CAMO processes the organization's structured data into metadata, meaning the data that is correlated to discover the relationships among the multiple data sets. The metadata is integrated into applications that allow organizations to discover knowledge or establish decision support systems that match the business process.
Custom applications are then built to use the metadata that generate explicit knowledge, which is displayed in a graphic format. Individuals then extract the visual reports to learn and add their own intelligence to the knowledge base. Through collaboration, the explicit knowledge is shared by the individuals and circulated throughout the organization, becoming tacit, or deeply embedded and understood.
As the organization accepts the knowledge, they contribute to it, according to their own knowledge and expertise of the business process. This creates a multiple increase in intelligence as people from throughout the organization learn and contribute to the knowledge core.
Data warehousing, great analytics or extensive data collection are part of the process but not what determines the success of a corporation's business intelligence investments. For maximum results, building a knowledge base is crucial. But without understanding how to process organizational data into knowledge, organizations run the risk of developing a knowledge management system that lacks the robustness and the attributes needed to gain considerable market share in the general marketplace.
Dave Lundahl welcomes feedback and conversation. He can be reached at david.lundahl@camo.com
Founded in 1984, CAMO develops process optimization software, knowledge discovery and decision support solutions. The company has 200 employees and more than 1,800 customers in 46 countries in the food, chemical, pharmaceutical and manufacturing industries. Headquarters are in Woodbridge, NJ with additional offices in the United States, India, the United Kingdom and Norway.For further information, please visit our web site at CAMO
CAMO, helping Smart People Get Smarter