What technologies are KM professionals buying?

An IDC survey of KMWorld attendees

By Susan Feldman, IDC

The preponderance of information within most organizations is unstructured--text, for the most part, but also audio, video and graphics files. Several years ago, IBM (ibm.com) estimated that upward of 75% of enterprise information was in those formats. How are organizations handling that information so that they know where it is and how to get at it? Are centralized content management systems the norm? What percentage of enterprises have implemented search, and for what purpose? How successful are knowledge workers in finding the information that they need?

Because content management forms the infrastructure for knowledge management, IDC surveyed attendees of the KMWorld 2001 Conference and Exposition in October to find out what technologies had been implemented or were planned in the near future. We also asked about searching behavior and success rates, and about what they were using knowledge management and content management for. Because attendance was down due to travel restrictions after Sept. 11, the number of responses was small, but still indicative of some trends. Here’s what we found:

Questions asked

We asked respondents to tell us about their content management and knowledge management purchasing intentions. Then we asked them to rate major features and technologies for their importance. Those included: content management for publishing to the intranet; workflow; versioning and rollback; search engines; alerting; categorization; taxonomy building tools; indexing and metatagging tools; image and audio search; text mining, single point of access for all materials in the enterprise; rich media asset management; streaming; digital rights management; wireless and mobile access; multilingual content management and multilingual content access.

In addition, we asked them to tell us how they were using content and knowledge management, and finally, we asked them to tell us how successful they were in finding the information for which they were looking.

Uses of CM and KM systems The primary use that respondents said they were making of both content and knowledge management initiatives was to find information. Fifty-four percent said that this was their primary use. That fits with the kinds of technologies that they have invested in: primarily basic publishing to the intranet and search engines. Web publishing follows with a distant 40%. Both of those are basic uses to which a CM or KM system could be put, and they may indicate that CM and KM are still in the early stage of their evolution.

Dreams, however, abound in the responses we received about future plans. Fifty-four percent plan to use the systems for training. Forty-four percent said that they will use it to track and manage documents for all purposes. Another 44% expected to use CM and KM for customer relationship management. And 40% expect to use the systems to provide project workspace, a collaborative application.

What features are important today?

Content management and search are the two bookends in the spectrum of technologies necessary for building an enterprise information system. The first creates repositories, and the second provides the access to them.

Last year, 32.3% of respondents to a KM survey reported that they had purchased search engine software, and an additional 36.3% planned to purchase search software in the coming year. This year, search was the most important feature of those listed, according to respondents. Sixty-nine percent of the respondents had implemented a search engine, or planned to implement one in the next 12 months. And 65% of them rated the search engine as being very to critically important. Apparently the buying plans and implementation projects reported a year ago were carried out, to give us the 69% installed base that we see at the end of 2001.

Content management for publishing to the intranet came in second, with 63% of respondents having implemented or planning on implementing it. Fifty-nine percent rated that function as very to critically important. That compares to 39% who had implemented content management a year ago, with an additional 49.9% planning to do so at the time.

Personalization and alerting, techniques for making sure that the right information gets to the right people, but also for trying to stem the information flood by removing extraneous information, appear to be gaining in importance. While only 20% reported that they had implemented personalization and alerting so far, a further 34% expected to implement it in the next 12 months, giving us a total of 54% who should have it by the end of the year. Forty-two percent rated those features as very or critically important.

A full comparison of 2001 data with a new 2002 survey will be published by IDC later this year.

Few implementations but a lot of interest

Comparing implementations with buying intentions is useful in determining emerging trends. Several of the technologies we asked about fall into the category of not popular now, but of growing importance in the next year or two. Surprisingly, workflow, a staple of the document and content management application, appears to fall into this category. It is possible that enterprises have been using minimal content management solely for the purpose of Web publishing, and have only begun to realize that the publishing process can get out of hand if a Web site is large, when pieces of it are scattered around the enterprise. Workflow ensures that the correct version is being worked on by all, that the people in the process are not working at cross purposes or duplicating each other’s efforts. It tells administrators where in the process a specific document can be found, and whether it is likely that schedules will be met. Whatever the reason for its current low use (only 15% said that they had implemented it), interest in workflow is high. Forty-one percent intended to implement it, and 43% rated it as very important.

A puzzlement

IDC believes that good information access requires several building blocks in addition to putting documents in a repository. One of the goals of the survey was to discover whether enterprises were investing in improving information finding by adding categorization, taxonomies and metadata tagging, It appears that to some extent, those technologies have been implemented. Fifty-two percent metatag and index their documents. Forty-seven percent categorize them. But only 38% have created or are using taxonomy building tools. It is possible that those processes are more widespread, but that they are being performed manually.

The tools to aid in categorization, metadata creation and taxonomy creation, however, are reasonably well implemented. What is puzzling is that their importance is not rated highly. Yet, most retrieval experts point to categorization, subject and other metadata assignment, and the use of taxonomies as essential to improving search and navigation. In particular, taxonomies can be used to create a browsing mechanism for those who want to know what is in a collection, or for those who don’t have enough terminology to create a good query. Categorization was ranked as important by only 38% of the respondents. Taxonomy building, by only 35%, and metadata creation only by 38% as well.

However, IDC believes that the emphasis on improving information finding will grow in the next three years. This baseline survey will be used to determine if in fact that emphasis does change.

Not interested

Not surprisingly, new technologies that seem ancillary to enterprise information management were minimally implemented, and ranked low in importance. This group included rich media asset management, image search, streaming technologies, text mining, digital rights management, and wireless and handheld access to enterprise information.

If we match the lack of interest to new products that vendors are beginning to offer, we see the greatest disparity of demand vs. supply. Vendors are preparing their products to be able to handle all types of media, not just text. They are adding specialized rich media asset management capabilities as well. Wireless access is driving more interest in XML tagging. Text mining is a hot area in product development. Digital rights management should be of prime importance in the wake of the Napster (napster.com) controversy. It is possible, however, that enterprises and other organizations are only now starting to consider those newer technologies. Perhaps as they become available in standard content management and search applications, they will gain more visibility.

Text mining is another puzzling area. If enterprises are snowed under with too much information, or if they are intent on improving their competitive intelligence, then text mining would seem to be an answer to those problems. Possibly the technology is too new and not well enough understood for enterprises to consider investing in it. Or perhaps they are waiting for text mining to become a part of a larger suite of information finding tools.

Effectiveness

This list of technologies represents a sizable investment for the enterprises and organizations that responded to the questionnaire. The question is, if the purpose of managing and publishing information is to make it accessible, have these technologies as they have been implemented been successful in connecting people to the information that they need? For that reason, we asked respondents to tell us how successful they considered themselves to be in finding information.

Only 21% of searchers considered themselves to be successful 85% or more of the time. And 38% felt they were successful less than 50% of the time. In fact, that means that 68% of the respondents said that they succeeded in finding what they needed less than 85% of the time. Yet they are probably knowledge workers who are more information-savvy than most. Their job titles included CEO, IT director, Information center director or library director, CIO and CFO.

Moreover, 39% spent more than nine hours a week searching, and 69% spent five hours or more a week searching. That is an enormous amount of time invested in fruitless searching. The cost to enterprises is staggering. If those highly paid professionals cannot find the information they seek, and if they are wasting that much time looking for information, then their productivity has gone down. Worse, the decisions that they are trying to make may be backed by incomplete information. Each of those failed searches is an information disaster in the making.

So what does this all tell us? Organizations are publishing their information to their intranets and to the public Web. They are searching for that information, and they are not finding it. Because retrieval is an extremely complex technology, the investments that have been made in technologies that might improve search have probably been undervalued. One has to wonder if categorization and metatagging has been tied into information finding to sharpen relevance ranking or to create browsing capabilities. Like data warehousing, it is possible that the rush to store and to publish has eclipsed the greater need to find. The foundation technologies that will enable better information seeking and finding are less prevalent than the repository building and search technologies. In the next five years, we can expect that the situation will change, and that the elements that create better search will receive the attention and the investment that they deserve.

Susan Feldman is research VP, Content Management & Retrieval Software Research, IDCwith IDC (idc.com), e-mail sfeldman@idc.com