Skip to main content
Generalized Error Path Algorithm
Pattern Recognition
  • Bin Gu, Nanjing University of Information Science & Technology & Mohamed bin Zayed University of Artificial Intelligence
  • Charles X. Ling, University of Western Ontario, Canada
Document Type

Model selection with cross validation (CV) is very popular in machine learning. However, CV with grid and other common search strategies cannot guarantee to find the model with minimum CV error, which is often the ultimate goal of model selection. Recently, various solution path algorithms have been proposed for several important learning algorithms including support vector classification, Lasso, and so on. However, they still do not guarantee to find the model with minimum CV error. In this paper, we first show that the solution paths produced by various algorithms have the property of piecewise linearity. Then, we prove that a large class of error (or loss) functions are piecewise constant, linear, or quadratic w.r.t. the regularization parameter, based on the solution path. Finally, we propose a new generalized error path algorithm (GEP), and prove that it will find the model with minimum CV error in a finite number of steps for the entire range of the regularization parameter. The experimental results on a variety of datasets not only confirm our theoretical findings, but also show that the best model with our GEP has better generalization error on the test data, compared to the grid search, manual search, and random search.

Publication Date
  • Cross validation,
  • Error path,
  • Model selection,
  • Solution path

IR deposit conditions:

  • OA (accepted version) - pathway b
  • 24 months embargo
  • Must link to publisher version with DOI
Citation Information
B. Gu and C. X. Ling, “Generalized error path algorithm,” Pattern Recognition, vol. 120, p. 1-12, Dec. 2021, doi: 10.1016/J.PATCOG.2021.108112.