The Professionalization of Search and Review in E-Discovery
In recent months, we have witnessed the evolution of document review and analysis, otherwise known as "search," in the context of legal discovery and electronic information management. Courts and practitioners have recognized that the practice of search in e-discovery is a complex undertaking that requires expertise beyond the realms of law and IT. This recognition should come as no surprise, as search—or information retrieval as it is formally known—is well established as a distinct academic field which draws from computer science, mathematics, linguistics and statistics. As applied to e-discovery, information retrieval assumes significant legal and IT dimensions, but it remains a distinct discipline.
This shift in the understanding of document review is unsettling for many. Practitioners worry about defensibility. Corporations worry about cost. Stakeholders are assessing the benefits and pitfalls, and are concerned about standards that might result. (With respect to standards, the National Institute of Standards and Technology’s TREC Legal Track is drawing considerable attention with its efforts to develop performance measurement protocols for e-discovery search methods.)
Addressing these concerns might start with the recognition that this shift in conceptual understanding reflects a "professionalization" of large-scale document review in legal undertakings. All technical fields experience a transition from experimentation and unconstrained practice to professional maturity; this is true of medicine, of engineering and, indeed, of the field of law itself. Like every professional community, information retrieval has academic centers, publications and conferences, and has spawned a large number of commercial applications, from Web search to legal electronic document review.
Historically, professionalization has served consumers well. It has resulted in greater efficiencies, greater quality, lower costs and reliable standards that can be trusted by consumers and society at large. The same benefits can be expected from the professionalization of legal search, or document review.
The Role of Information Retrieval
In the context of enterprisewide electronic information management, information retrieval can help IT transition from primarily dealing with the storage of information to becoming a strategic partner for corporate records retention. Information retrieval offers principled methods to segregate relevant from irrelevant documents to a desired level of scientifically established accuracy. As a result, it can reduce the enormous hardware and software costs expended to store electronic information, but also supplant the practice of relying on employees to make retention decisions, with the inconsistency and typical "overcapture" this entails.
For a corporation engaged in litigation, information retrieval experts can help improve culling collections by eliminating documents from review without significant loss of relevant material. They can also select technologies to help execute review; they can design and administer the overall review process, and they can even select automated document review methods that entirely replace human review.
Information retrieval experts can also harness attorneys’ subject matter knowledge and apply it in a principled, scientific manner to a range of search tasks. Do documents exist that substantiate the presence or absence of certain activities? Did a deponent know of certain facts at a given time? What other custodians might be relevant to a given topic? Does a production reveal certain behavioral patterns? How do I narrow down all documents relevant to tomorrow’s depositions to a hundred documents that I can thoroughly analyze? Information retrieval contributes principled approaches to answering these questions and, in so doing, helps litigators win their cases.
If needed, information retrieval experts can also help attorneys and corporations explain the methods utilized in document retention and production processes, by answering the basic questions that might be asked of a producing party: how accurate was your litigation hold or production-review process, and how do you know? Information retrieval experts can provide the process knowledge that is necessary to successfully withstand defensibility challenges.
Developing Standards: TREC
The recognition of document review as a professional field leads to the question of standards: if information retrieval in the legal arena is a profession, by what standards will consumers assess this profession? The answer seems to be the Text Retrieval Conference (TREC) protocols. TREC recently developed the "Legal Track Interactive Task" designed to replicate document review in productions and to measure the accuracy of review methods. The evaluation is open to any document review provider, from law firms to technology vendors to professional services organizations.
TREC offers unbiased information: when a service provider makes representations as to the accuracy of their approach or technology, TREC serves as the equivalent of the Insurance Institute for Highway Safety in that a vendor’s claims can be independently verified. It is unlikely that TREC will become a certification entity or standard-setting body, but its evaluation forum will inevitably become part of future certification efforts in the legal information retrieval field. This will serve practitioners and the courts well. Scientifically meaningless claims of methods or technologies achieving "99% accuracy" will be debunked, and corporations will be less likely to suffer the consequences of seeing those claims debunked in court.
Information retrieval expertise can help reduce both IT costs and litigation costs, while enhancing litigation readiness and reducing risk. In doing so, it maximizes the strategic value of the IT investment across the enterprise. As credible measurement protocols such as TREC’s are more widely adopted, a range of well-performing methods will demonstrate their performance. These methods, as academic evidence shows, will not necessarily be perfect, but they will be reasonable (in both the common and legal sense of the term), enabling the efficiencies, cost-reduction and defensibility that IT groups and attorneys need and that courts increasingly expect.
H5 provides a full range of advisory and implementation services to help law firms and corporations meet the practical challenges of finding, assessing and managing information throughout the lifecycle of their legal matters. H5 has offices in New York City and San Francisco. Visit
www.h5.com to learn more.