Opinion
21-cv-03958-JSW
02-23-2023
ALIVECOR, INC., Plaintiff, v. APPLE, INC., Defendant.
ORDER DENYING MOTION FOR RELIEF FROM NON-DISPOSITIVE ORDER RE: DKT. NO. 140
JEFFREY S. WHITE UNITED STATES DISTRICT JUDGE
This matter comes before the Court upon consideration of Defendant's motion for relief from a non-dispositive discovery order. The Court has considered the parties' papers, relevant legal authority, and the record in this case, and it DENIES Defendant's motion.
BACKROUND
This dispute arises out of the parties' decision to utilize technology-assisted review (“TAR”) to locate and produce ESI material in response to requests for production. In brief, TAR is a method by which an algorithm is created to “enable [a] computer to determine relevance, based on interaction with (i.e., training by) a human reviewer” to search for and produce ESI based discovery. DaSilva Moore v. Publicis Groupe, 287 F.R.D. 182, 184 (S.D.N.Y. 2012); see also Timothy T. Lau and Emery G. Lee, III, Technology-Assisted Review for Discovery Requests, A Pocket Guide for Judges at 1, Federal Judicial Center (2017). Two of the key metrics used in TAR are “recall” and “precision.” “Recall is the fraction of relevant documents identified during a review; precision is the fraction of identified documents that are relevant. Thus, recall is a measure of completeness, while precision is a measure of accuracy or correctness. The goal is for the review method to result in higher recall and higher precision than another review method, at a cost proportionate to the ‘value' of the case.” Da Silva Moore, 287 F.R.D. at 189-90 (citing Maura R. Grossman & Gordon V. Cormack, Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, Rich. J.L. & Tech. Spring 2011, at 8-9).
The DaSilva Moore opinion contains a more detailed discussion of how a party would create the search algorithm, which is drawn from an article written by the author of that opinion. 287 F.R.D. at 184 (quoting Andrew J. Peck, Search, Forward: Will manual document review and keyword searches be replaced by computer-assisted coding?, L. Tech. News, Oct. 2011, at 25, 29).
The parties entered into a stipulated protocol for discovery of ESI (“ESI Protocol”). The ESI Protocol references the possibility that a party might use TAR. (Dkt. No. 45, ESI Protocol at 3.) After the parties could not agree on a deadline to substantially complete their document production, Judge Kim set a deadline of October 15, 2022, which was subsequently extended to October 31, 2022.
Defendant argues that although Plaintiff has certified its document production is substantially complete, Plaintiff overproduced non-responsive and likely underproduced responsive documents. Defendant attributes this to a flawed TAR algorithm. Plaintiff responds that it structured its TAR process to be conservative. In its view, Defendant's complaints stem in part from overbroad search terms and discovery requests.
The parties submitted the dispute to Judge Kim. On January 18, 2023, Judge Kim issued a summary denial of Defendant's motion. (Dkt. No. 135 (“Discovery Order”).)
The Court could refer this matter back to Judge Kim to clarify her reasoning. However, fact discovery is set to close on February 27, 2023. In addition, based on the number of discovery disputes that have been raised, it seems likely the Court would be called upon to review a subsequent order. Accordingly, in the interest of judicial economy, the Court will address the dispute without a referral back to Judge Kim.
ANALYSIS
When a party objects to a non-dispositive order issued by a magistrate judge, a district court “must consider timely objections and modify or set aside any part of the order that is clearly erroneous or is contrary to law.” Fed.R.Civ.P. 72(a); see also 28 U.S.C. § 636(b)(1)(A). Defendant argues the Discovery Order is clearly erroneous. (Mot. for Relief at 2:8-9.) A ruling is clearly erroneous if the reviewing court, after considering the evidence, is left with the “definite and firm conviction that a mistake has been committed.” United States v. U.S. Gypsum Co., 333 U.S. 364, 395 (1948).
A decision is contrary to law if the magistrate judge fails to apply or misapplies relevant case law, statutes, or rules of procedure. See, e.g., United States v. Cathcart, 2009 WL 1764642, at *2 (N.D. Cal. June 18, 2009).
Rule 26 requires that a party must supplement a discovery response if that “party learns that in some material respect the disclosure or response is incomplete or incorrect, and if the additional or corrective information has not otherwise been made known to the other parties during the discovery process or in writing[.]” Fed.R.Civ.P. 26(e)(1)(A). In their ESI Protocol, the parties agreed that if they used TAR, they would meet and confer on an appropriate protocol. (ESI Protocol at 3 n.1.) The parties exchanged drafts of such a protocol and agreed upon a recall rate of 70%. Defendant asserts it conducted “several tests to assess the accuracy of [Plaintiff's] TAR algorithm,” which it argues showed the “algorithm cannot reliably distinguish between responsive and unresponsive documents.” (Dkt. No. 133, Motion to Compel at 2:2-9.) Defendant did not support those assertions with evidence, such as a declaration from the attorney who performed the test. Defendant also has not introduced examples of documents from those tests that it contends should have been flagged as responsive but were not. See, e.g., Youngevity Int'l Corp. v. Smith, No. 16-cv-00704-BTM (JLB), 2019 WL 1542300, at *11 (S.D. Cal. Apr. 9, 2019); Winfield v. City of N.Y., No. 15-cv-05236 (LTS) (KHP), 2017 WL 5664852, at *5 (S.D.N.Y. Nov. 27, 2017).
To the extent disputes remained, Defendant could have - but did not - bring disputes about the protocols to Judge Kim's attention.
Defendant also argues the relevant time period is between September 2018 and August 2019 and asserts that Plaintiff produced documents outside of that time period. However, at least some of Defendant's request for production seek documents dating back to 2010. (See, e.g., Mot. to Compel, Ex. 1, Requests for Production (“RFP”) Nos. 30-34.) Other RFPs do not include any specific time frame. (See generally id.)
“[P]erfection in ESI discovery is not required; rather a producing party must take reasonable steps to identify and produce relevant documents.” Winfield, 2017 WL 5664852, at *9; accord Reinsdorf v. Skechers U.S.A, Inc., 296 F.R.D. 604, 615 (C.D. Cal. 2013) (“[W]hile parties must impose a reasonable construction on discovery requests and conduct a reasonable search when responding to the requests, the Federal Rules do not demand perfection.”). The Court also finds the following discussion from Winfield apt:
[T]here is nothing so exceptional about ESI production that should cause courts to insert themselves as super-managers of the parties' internal review processes, including training of TAR software or to permit discovery about such process, in the absence of evidence of good cause such as a showing of gross negligence in the review and production process, the failure to produce relevant specific documents known to exist or that are likely to exist, or other malfeasance.Winfield, 2017 WL 5664852, at *9. Having considered the record before Judge Kim, the Court concludes her ruling was not clearly erroneous and DENIES Defendant's motion for relief.
IT IS SO ORDERED.