How do you measure success?
A successful KM system contributes to achieving the goals of the enterprise while enhancing job performance and increasing the job satisfaction of each member of the organization. By demonstrating that knowledge management is improving performance, reducing costs and increasing customer responsiveness, the organization's leadership will see the explicit link between KM and the success of their business strategy. That will lead to a desire by leadership to better understand KM and to ensure that it is properly resourced.
The work force will want to participate in knowledge management activities if management provides a venue that allows employees to get advice and information directly related to their immediate tasks and responsibilities. That leads to active contribution to the organization's knowledge repositories and the regular use of its KM capabilities. Recognizing that those cause-and-effect relationships exist is a critical step in shifting organizational culture toward one of knowledge sharing, capture, reuse and innovation.
That being said, how can you determine if your KM system is successful? We do it through a multifaceted set of metrics that evaluate the system from the same three cause and effect perspectives. They are:
- achieving organizational goals through outcome metrics and leading indicator metrics,
- providing value to individuals through performance metrics, and
- evaluating KM readiness and maturity through KM learning metrics.
Achieving organizational goals
A major obstacle in implementing knowledge management is demonstrating that investments in KM are providing value to the organization. Most of us have encountered the skeptical executive who, after the KM presentation, leans back in the chair and with folded arms says those fateful words: "Show me the ROI." And you think to yourself, "How can I ROI an attitude?"
One of the ways to describe ROI is through the use of analogy or benchmarking. We have found that managers who are focused on ROI numbers are not usually persuaded by the success stories of others. We prefer to show the cause-and-effect linkages between KM investment and an organization's business goals (Figure 1 (See P.28, July/August KMWorld 2003, Vol.12, Issue 7)) by using Kaplan and Norton's balanced scorecard concept. That framework matches our systems perspective (every thing is related to everything), and provides an evaluation and tracking mechanism to continuously monitor implementation of the organization's strategy and to indicate adjustments. The balanced scorecard incorporates both outcome and leading indicator metrics.
Outcome metrics evaluate the attainment of organizational goals and objectives. They are used to evaluate past performance at both the enterprise and community level. There is no standard set of outcome metrics. Each organization must develop a tailored set that addresses its individual needs. In the balanced scorecard approach, the authors place the financial perspective at the top of the objectives/metrics hierarchy. The perspective is the value proposition for the stockholders of for-profit organizations. In order to translate stockholder value for many of our government clients, we had to rethink what constitutes value and who are the stockholders.
Every government organization has a mission that defines what it is chartered to do. Any inability to perform that mission degrades stockholder value. As an organization develops its strategy, it identifies performance gaps and establishes a plan to overcome those gaps. Therefore, performance optimization or change in deficiency is a key element in determining value, and can be used by government organizations to replace financial metrics.
Choosing outcome metrics is not a trivial task. It requires a great deal of hard work and thought. Pay heed to what Jamishid Gharajedaghi said: Winning is fun; in order to win you have to keep score, and the way you keep score defines the game.
Leading indicator metrics evaluate the environmental influence on the organization and its KM system. They are used to evaluate the future viability of the organization's KM strategy. Using the vernacular of strategic planning, performance drivers can be compared to the strengths and weaknesses listed on a SWOT (strengths, weaknesses, opportunities, threats) analysis. We know that strengths remain strengths and weaknesses remain weaknesses as long as there are no changes in the organization's internal and external environment. Leading indicators are used to continually monitor the environment to detect what Karl Albrecht ("The Northbound Train") refers to as a "Tsunami Event"--an event so significant that it could restructure the way that the organization does business. Those metrics address the key question, "What could happen that would invalidate my strategies, plans and implementation actions?"
Providing value to individuals
Performance metrics evaluate the satisfaction of KM technology users. Organizations can invest significant resources in infrastructure and tools to facilitate knowledge management across the enterprise. Too many of us have experienced the installation of some potentially valuable software tool that never seems to get used. Rejection occurs for a variety of reasons. Sometimes the tools are too complicated, sometimes they fail to make the user's life simpler and sometimes they lack that intangible eye appeal. We use three basic categories of metrics for evaluating collaboration KM workspaces for meeting KM objectives: (1) usage statistics, (2) business process and outcome metrics, and (3) KM workspace usability metrics. The following list contains the metrics within those categories that form our default set:
- Usage--number of register users/user accounts, frequency of use.
- Business value--improved response to rapid changes; process improvement/efficiency; outreach/expansion of contributions; value of knowledge mined from community usage, exchange and input of CoP via the virtual CoP KM workspace.
- Usability--ease of use/confidence of users, adequacy of user support and training, KM workspace performance in meeting requirements, overall user/customer satisfaction rating. KM learning metrics evaluate the organization's KM training program. The degree by which knowledge management is uniformly practiced across the organization depends on how well its members use common KM concepts, principles and employment techniques. That is not something that happens by accident. A mature, KM-focused organization will make training and education part of its business strategy.
Cynthia J. Odom, MSOD, is a principal organizational improvement specialist at Keane Federal Systems. John F. Starns, D.Sc., is a principal consultant for Knowledge Management Strategy and Plans Integration at Northrop Grumman Information Technology. Both are adjunct professors of engineering and knowledge management at The George Washington University, e-mail odomc@cox.net, jfstarns@cox.net.