Relativity Assisted Review Parameters

kCura recommends not using Assisted Review, or TAR, for data sets less than 50,000 documents. See page 10 of the Assisted Review Guide for version 9.5 of Relativity here. Compare this the Catalyst Predict system which can apparently employ TAR on sets as small as 8,000 documents. See here. Also note the kCura does not recommend using Assisted Review on document populations that consist largely of numbers - presumably those with a high percentage of spreadsheets.

TAR For Smart People Outline - Chapters 13 and 14

Here's the last installment in my outline of John Tredennick's 'TAR for Smart People'. I last posted an installment on February 10, 2017. This night's installment is on Chapter 13, 'Case Study: Using Insight Predict for Review of Rolling Opposing Party Productions' and Chapter 14, 'Case Study: TAR Does Double Duty in a Government Probe'.

13. Case Study: Using Insight Predict for Review of Rolling Opposing Party Productions Insight Predict Finds 75% of Hot Docs While Cutting Review 92%

A. Challenge: Find Hot Documents in Opponent’s Rolling Productions - Prioritized Review with Insight Predict

1. Insight Predict - Uses Continuous Active Learning to rank each new document as it is added.

2. Insight Predict can handle document productions with a low richness.

3. New ranking rounds were run every 10 minutes. Richness increased from 1% to 7%. 70% of hot documents found after reviewing only 8% of production.

B. Contextual Diversity vs. 'Hide the Ball'

1. Samples the production to insure there are no pockets of documents that escape human review. A pocket of new documents unlike anything reviewers have seen before is immediately recognized, and exemplars from those new pockets will be pulled as contextual diversity seeds and put in front of reviewers in the very next batch of documents to be reviewed.

C. Bottom Line

1. Majority of documents that had to be reviewed was reduced by more than 90%.

14. Case Study: TAR Does Double Duty in a Government Probe Insight Predict Reduces Review and Rescues Privileged Documents

A. Challenge: Review Carefully But Control Time and Cost

Government investigation required the review of 60K documents. Initial sample put richness at 20%. Review stopped at 60% of total population but got 96% recall.

B. Using Predict as a Check for Privileged Documents

1. Documents already ranked for privilege used as training seeds. Batched top 100 documents that Predict ranked as privileged but reviewers ranked as responsive. From this batch review team only found 5 privileged documents. Process continued several more times.

2. Only 500 documents reviewed before no more additional privileged documents, and only seven more privileged documents identified.

3. 40% of document production eliminated from linear review

TAR for Smart People Outline - Chapters 11 and 12

Here's another installment in my outline of John Tredennick's 'TAR for Smart People'. I last posted an installment on January 27, 2017. This night's installment is on Chapter 11, Case Study: Using TAR to Find Hot Docs for Depositions: How Insight Predict Succeeded Where Keywords Failed, and Chapter 12, Case Study: Using TAR to Expedite Multi-Language Review.

11. Case Study: Using TAR to Find Hot Docs for Depositions: How Insight Predict Succeeded Where Keywords Failed

A. Challenge: Quickly Find Hot Docs in Garbled Production

1. Multi-district Litigation involving medical device. 77K electronic documents produced shortly before depositions. No meta data, and poor scanning leads to bad OCR. Results of focused searches consist of only 5% being possible deposition exhibits, and 46% relevant.

B. Solution: Using Insight Predict to Prioritize Hot Documents for Review

1. Lead attorney QC'd documents already tagged as hot, and few hundred more targeted hits and samples.

2. Using those seeds ranked the entire document set for hot documents.

3. Top 1000 documents then pulled for attorneys to evaluate.

4. TAR increases docs being potential exhibits to 27% and relevant docs to 65%.

C. Good Results from Difficult Data

1. Insight Predict allows you to use an unlimited number of seeds from judgmental sampling.

12. Case Study: Using TAR to Expedite Multi-Language Review: How Insight Predict's Unique Capabilities Cut Review by Two-Thirds

A. The Challenge: Quickly Review Multi-Language Documents

Shareholder class action regarding violations of securities laws requires review of mixed set of Spanish and English documents. Review 66K files for responsive documents to produce.

B. The Solution: Prioritize Documents for Review Using Insight Predict

1. Few hundred emails already reviewed by attorneys that came from key custodians used as seeds to train Predict engine.

2. Separate rankings for the two languages. On demand batches sent to review team from top of the ranking round.

3. After training predict with initial seeds responsiveness of batches sent to review team increased from 10% to 40%.

4. 91% recall achieved after reviewing only 1/3 of documents.

C. Uncovering a Hidden Trove

1. Contextual Diversity Sampling - Predict looks for biggest set of documents that have no reviewer judgments, finds best examples in them, and send to review team.

2. In case study contextual diversity sampling found hundreds of spreadsheets omitted from initial manual seeds.

3. Every responsive document sorted to the top third of the review set, making review of two-thirds of set unnecessary.