December 06, 2010, 4:50 PM — Document dump." "Unsearchable morass." That's how Ontario Superior Court Justice Cary Boswell described the nearly 23 million pages of electronic records handed over by the prosecution in an ongoing criminal fraud case of three former Nortel Networks executives.
During a hearing last December, defense lawyers argued that the sheer amount of material provided on a hard drive -- the equivalent of 8,000 to 10,000 boxes of paper -- was so "staggering" and disorganized that it couldn't be effectively searched for information that might help the defendants.
In a ruling afterward, the judge agreed, ordering the prosecution to "re-disclose" any relevant material in a more organized fashion.
That case offers an example of the challenges legal professionals face with e-discovery. And those difficulties are compounded by the fact that typical computer searches don't find all of the relevant information in a data dump. For example, tests by the Text Retrieval Conference (TREC), an international workshop that assesses various information retrieval approaches, show that Boolean keyword searches locate only 22% to 57% of the total number of relevant documents.
"If you're conducting searches electronically, you're never going to be able to say that you turned over every stone," warns Susan Wortzman, a partner at Wortzman Nickle PC, a Toronto law firm specializing in e-discovery.
One problem with today's search tools is the prevalence of false positives. Keyword searches retrieve all the documents containing a specified term, regardless of context. The result is a collection of files that are often irrelevant to a legal team. For example, a search for the word record could turn up documents related to a Beatles album, a Guinness world record or a recorded message.
But that's not all. Documents containing inadvertent misspellings can easily fall through the cracks of a standard keyword search. Then there's the inherent ambiguity of language, the combination of text and images, and the introduction of errors by optical character recognition software -- all factors that can significantly impair the e-discovery process.
"Keyword searching is a blunt instrument," laments Patrick Oot, co-founder of the nonprofit Electronic Discovery Institute and former director of electronic discovery at Verizon Communications Inc.
The Human Factor