Featured Expert Column –Judicial Gatekeeping of Expert Evidence
Evan M. Tager, a Partner in the Washington, DC office of Mayer Brown LLP, with Carl J. Summers, an Associate with Mayer Brown LLP.
The US Court of Appeals for the Third Circuit recently rejected an attempt to substantially lower the standard for admission of expert testimony resting on studies that have not produced replicated and statistically significant findings. Specifically, the plaintiffs in In re Zoloft Products Liability Litigation argued that the district court erroneously imposed a rigid, bright-line rule that an expert must present replicable, statistically significant findings. The Third Circuit held that the district court had not established such a bright-line rule, but rather had made a factual finding that teratologists—scientists who study abnormalities in human development—“generally required replication of significant results.” After dispensing with the plaintiffs’ flawed interpretation of the district court’s decision, the Third Circuit affirmed the exclusion of the expert testimony on the ground that the expert had selectively chosen data that supported his opinion and inconsistently applied his methodology, thus rendering his opinions unreliable.
The plaintiffs in this mass action alleged that Zoloft—a prescription drug manufactured by Pfizer that is used to treat depression and anxiety—causes cardiovascular birth defects when used in the early stages of pregnancy. Because ethical concerns prohibit double-blind, randomized studies on pregnant women, research on birth defects must rely on less rigorous observational studies. As noted by the district court, common scientific practice dictates that, even when a correlation has been found to be statistically significant within a narrow confidence interval, a single study remains insufficient to establish causation given the potential for random error, bias, confounding variables, or some other flaw with the study. Accordingly, scientists look to whether the results of an observational study have been replicated to determine whether causation exists.
In the Zoloft litigation, the plaintiffs sought to qualify Dr. Nicholas Jewell as an expert witness. Dr. Jewell is a statistician; he does not purport to be an expert in embryology or cardiology. According to Dr. Jewell, the results of eleven published, peer-reviewed epidemiological studies and his own re-analysis of those studies establish a link between Zoloft and cardiovascular birth defects.
After conducting a Daubert hearing, the district court concluded that Dr. Jewell could not testify as an expert. Reviewing the relevant studies, the district court concluded that there were no replicated statistically significant findings from non-overlapping data—that is, confirming results from studies that do not rely on the same populations or underlying studies. The district court also questioned why Dr. Jewell used only certain studies—but not the entire universe of relevant studies—in conducting his meta-analysis. The district court further concluded that Dr. Jewell failed to consistently apply his own methods and standards. It explained that “Dr. Jewell’s selective emphasis on trends and general consistency only when such concepts support his opinion is one example of ‘situational science’ which renders his opinion unreliable.” Based on these findings and Dr. Jewell’s inconsistent application of his own methods, the district court excluded his expert testimony.
In reviewing that ruling, the Third Circuit emphasized that the “central” “question” was “whether statistical significance is necessary to prove causality.” The court rejected the plaintiffs’ contention that the district court improperly established a rigid, bright-line rule that every expert must present replicable, statistically significant findings. The Third Circuit noted that such a “bright-line rule” on that question would be inappropriate because “the requisite proof necessary to establish causation will vary greatly case by case.” As the Third Circuit saw it, “the District Court was not creating a legal standard, but merely making a factual finding” that “teratologists generally require replication of significant results.” Considering the replicability of findings as a central factor in the admission of expert testimony, the Third Circuit explained, “did not prevent [the district court] from considering other evidence of reliability.”
The Third Circuit proceeded to affirm the district court’s exclusion of Dr. Jewell’s expert testimony. In doing so, the court focused on Dr. Jewell’s application of the “Bradford Hill/weight of the evidence criteria”—a set of nine principles that are often used to establish a causal relationship in epidemiological studies. The Third Circuit emphasized that, even though Bradford Hill criteria are generally reliable, they must be reliably applied and that any step that renders an analysis unreliable under Daubert renders the expert’s testimony inadmissible.
After conducting an in-depth analysis of the relevant studies, the Third Circuit found multiple flaws with Dr. Jewell’s methodology. For instance, Dr. Jewell declined—without explanation—to conduct a qualitative analysis that he acknowledged was possible in his report. The Third Circuit believed that this unexplained refusal indicated an unreliable application of Dr. Jewell’s stated methodology. The Third Circuit determined, moreover, that Dr. Jewell’s re-analysis of a study that had not found a statistically significant increase in the likelihood of birth defects was “conclusion-driven.” And finally, the Third Circuit criticized Dr. Jewell for “inconsistently discussing statistical significance” in analyzing the various studies.
The Third Circuit’s Zoloft decision rejected an attempt by the plaintiffs’ bar to open the floodgates to expert testimony based on un-replicated and statistically insignificant studies. In that sense, the decision is a resounding victory. To be sure, a bright-line rule mandating replicable, statistically significant studies in these cases would have been preferable, but the Third Circuit made clear that plaintiffs bear the burden of proving causality and that statistical significance and replicability are important factors in evaluating the admissibility of expert testimony on causation. And the Third Circuit’s emphasis on the importance of reliability at each step of the scientific inquiry—and its willingness to look beyond the expert’s conclusions to the underlying data and method of analysis—is a powerful antidote to conclusion-driven studies and inconsistently applied methods, as its rejection of Dr. Jewell’s testimony amply demonstrates. The Zoloft decision will thus provide a useful and persuasive citation about the importance statistical significance and reliable methods.