Skip to main content
Article
On Evaluating MHC-II Binding Peptide Prediction Methods
PloS ONE
  • Yasser EL-Manzalawy, Iowa State University
  • Drena Dobbs, Iowa State University
  • Vasant Honavar, Iowa State University
Document Type
Article
Publication Version
Published Version
Publication Date
1-1-2008
DOI
10.1371/journal.pone.0003268
Abstract
Choice of one method over another for MHC-II binding peptide prediction is typically based on published reports of their estimated performance on standard benchmark datasets. We show that several standard benchmark datasets of unique peptides used in such studies contain a substantial number of peptides that share a high degree of sequence identity with one or more other peptide sequences in the same dataset. Thus, in a standard cross-validation setup, the test set and the training set are likely to contain sequences that share a high degree of sequence identity with each other, leading to overly optimistic estimates of performance. Hence, to more rigorously assess the relative performance of different prediction methods, we explore the use of similarity-reduced datasets. We introduce three similarity-reduced MHC-II benchmark datasets derived from MHCPEP, MHCBN, and IEDB databases. The results of our comparison of the performance of three MHC-II binding peptide prediction methods estimated using datasets of unique peptides with that obtained using their similarity-reduced counterparts shows that the former can be rather optimistic relative to the performance of the same methods on similarity-reduced counterparts of the same datasets. Furthermore, our results demonstrate that conclusions regarding the superiority of one method over another drawn on the basis of performance estimates obtained using commonly used datasets of unique peptides are often contradicted by the observed performance of the methods on the similarity-reduced versions of the same datasets. These results underscore the importance of using similarity-reduced datasets in rigorously comparing the performance of alternative MHC-II peptide prediction methods.
Comments

This article is from PloS ONE 3 (2008): e3268, doi: 10.1371/journal.pone.0003268. Posted with pemrission.

Rights
This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Copyright Owner
EL-Manzalawi et al.
Language
en
File Format
application/pdf
Citation Information
Yasser EL-Manzalawy, Drena Dobbs and Vasant Honavar. "On Evaluating MHC-II Binding Peptide Prediction Methods" PloS ONE Vol. 3 Iss. 9 (2008) p. e3268
Available at: http://works.bepress.com/drena-dobbs/30/