A deletion/substitution/addition algorithm for classification neural networks, with applications to biomedical data

Blythe Durbin, Genentech, Inc.
Sandrine Dudoit, Division of Biostatistics and Department of Statistics, University of California, Berkeley
Mark J. van der Laan, Division of Biostatistics and Department of Statistics, University of California, Berkeley

Abstract

Neural networks are a popular machine learning tool, particularly in applications such as protein structure prediction; however, overfitting can pose an obstacle to their effective use. Due to the large number of parameters in a typical neural network, one may obtain a network fit that perfectly predicts the learning data, yet fails to generalize to other data sets. One way of reducing the size of the parmeter space is to alter the network topology so that some edges are removed; however it is often not immediately apparent which edges should be eliminated. We propose a data-adaptive method of selecting an optimal network architecture using a deletion/substitution/addition algorithm. Results of this approach to classification are presented on simulated data and the breast cancer data of Wolberg and Mangasarian (1990).