Signal in Noise: Evaluating Reported Reproducibility of Serum Proteomic Tests for Ovarian Cancer

Keith A. Baggerly, The University of Texas M.D. Anderson Cancer Center
Jeffrey S. Morris, The University of Texas M.D. Anderson Cancer Center
Sarah R. Edmonson, Baylor College of Medicine
Kevin R. Coombes, The University of Texas M.D. Anderson Cancer Center

Abstract

Proteomic profi ling of serum initially appeared to be dramatically effective for diagnosis of early-stage ovarian cancer, but these results have proven diffi cult to reproduce. A recent publication reported good classifi cation in one dataset using results from training on a much earlier dataset, but the authors have since reported that they did not perform the analysis as described. We examined the reproducibility of the proteomic patterns across datasets in more detail. Our analysis reveals that the pattern that enabled successful classifi cation is biologically implausible and that the method, properly applied, does not classify the data accurately. We show that the method used in previously published studies does not establish reproducibility and performs no better than chance for classifying the second dataset, in part because the second dataset is easy to classify correctly. We conclude that the reproducibility of the proteomic profi ling approach has yet to be established.