Electronic Discovery Institute Course - Class 19 - Measuring Results: Surpassing Legal Compliance
Here's a continuation of my postings about the Electronic Discovery Institute's online e-discovery certification program, that you can subscribe to for just $1. I last blogged about this program on March 11, 2018. Go to https://www.lawinstitute.org/ to sign up for it.
This course is taught by Debra Ruston, a litigation support professional with Freddie Mac; David Steele, the head of Home Depot's commercial litigation group; and Bennett Borden, a partner at Drinker Biddle LLP, who is also its Chief Data Scientist and co-chair of its electronic discovery and information governance group.
What Are Metrics?
Metrics measure the volume, speed and cost of the electronic discovery process. The law has developed to allow for argument on the basis of metrics.
Types of Metrics
Volume, speed, and cost are all interrelated. Tracking the three key metrics can ensure that a firm or company has sufficient resources to meet electronic discovery demands.
What Kind of Data Can Metrics Measure?
Volume Volume typically measures the number of files.
Time/Speed It is important to measure how fast a reviewer is going through documents.
Cost Cost is involved in accessing information. Files must be located on servers or thumb drives, which entails costs for people performing such searches. Looking at documents with review tools also incurs costs.
Why Are Metrics Important?
It's important to know what the scope of a project is. It's important to understand the effort involved in locating data. It used to be that parties would argue over search terms, or where to look for key files. With metrics parties can test search terms on statistically valid samples.
It may be advisable to review the metrics for previous projects. De-NISTing and de-duping will narrow down a data set to what is reviewable for production. The money required for the review of number of GBs may overshadow the value of case.
Recent law has promoted the use of metrics. It's possible to prove that something didn't happen through the use of metrics.
How Are Metrics Gathered? Metrics can be gathered by a project manager, but it may also be necessary to rely upon the services of a vendor. Usually a group of people will collect the metrics for an electronic discovery project. It may be necessary to calculate the number of documents each custodian will have.
The same terms may be used to describe factors that are quite different.
Using Metrics in the eDiscovery Workflow A client will always want to know the cost of case at the onset of the representation. Looking at metrics from previous matters lets you plan to control risks and estimate costs. Excel pivot tables can help show the implication of metrics.
At the beginning of a case, often very little is known outside what is described in a complaint. The number of people involved is directly proportional to the volume of data that will be involved.
Budgeting
The most usual metrics in terms of budgets are the cost for review per document or per GB. The cost per GB for processing is also important.
Collection
Metrics play a key role in the defensibility of a firm's process. Metrics will show just how rich a particular data source is. Metrics help with planning - how many people are needed; which collection methodologies are needed. A review of a subset of data may lead to the conclusion that a full review is not necessary.
Collection involves the gathering of data from email servers; file servers; and from within databases. It is important to have good metrics on the number of custodians that are relevant to a case.
Processing
Processing involves taking native files and transferring them into a form where they can be analyzed. Certain categories of documents may have to be processed at the beginning of a project.
The total volume of the data set; the number of reviewers necessary to review it; the amount of time that processing will take are all factors that need to be taken into consideration. There may be a significant number of password protected files that will take time to access.
Whether or not to bring processing in house will be dependent on whether or not a task has to be performed infrequently. A constant volume of data requiring processing may make in-house solutions make sense.
Determing Undue Burden
Metrics can be very important in determining whether or not it's necessary to go after certain types of data. There is a balancing test between the cost of getting the data and its importance to the resolution of a case. For example, it will be necessary to estimate the number of hours to will take to restore back-up tapes. If the cost of production outweighs the value of case, it may be necessary consider settling.
Review An attorney can only review so many documents in an hour. Advanced technologies will allow an attorney's decision about a particular document to be applied to many similar documents. Metrics focus on how fast reviewer can go over documents; and the quality and accuracy of their review. It is necessary to continuously track the number of documents that are reviewed.
Reviewers should use a workflow that allows them to prioritize certain types of documents. It is important to track culling decisions that second level reviewers disagree with. There is not necessarily a fixed benchmark for the number of documents reviewed per hour that is regarded as being efficient.
Production
Most discovery disputes will take place after the exchange of a production. Good metrics will allow for a lot of these disputes to be defused. It's important to track overall document counts and ensure that privileged documents are not included in productions. Metrics will assist with quality control, and prevent documents from being produced which should be withheld.
Vendors & Other 3rd Parties
The metrics may be dependent on billing arrangements. It's important to keep track of how many people are working on an individual case, and how long it takes to do certain tasks.
Uniform Task-Based Management Systems & Billing Codes
Standardized billing codes can be used to classify common tasks performed by attorneys. They provide a granular view of work performed on a project, and ensure that invoices are correct. L600 is the code used for electronic discovery. The codes also help ensure that the right people are performing the right tasks.
In-house attorneys often rely on billing codes.
Measuring Success
Quality and accuracy are the paramount considerations. Speed is not a key consideration in every case. A volume of data collected tells you something very different from a volume of data reviewed. Metrics should be reviewed to analyze a case on an ongoing basis.