Consistent selection of tuning parameters via variable selection stability

Wei Sun, Junhui Wang, Yixin Fang

Research output: Contribution to journalArticle

30 Scopus citations

Abstract

Penalized regression models are popularly used in high-dimensional data analysis to conduct variable selection and model fitting simultaneously. Whereas success has been widely reported in literature, their performances largely depend on the tuning parameters that balance the trade-off between model fitting and model sparsity. Existing tuning criteria mainly follow the route of minimizing the estimated prediction error or maximizing the posterior model probability, such as cross validation, AIC and BIC. This article introduces a general tuning parameter selection criterion based on variable selection stability. The key idea is to select the tuning parameters so that the resultant penalized regression model is stable in variable selection. The asymptotic selection consistency is established for both fixed and diverging dimensions. Its effectiveness is also demonstrated in a variety of simulated examples as well as an application to the prostate cancer data.

Original languageEnglish (US)
Pages (from-to)3419-3440
Number of pages22
JournalJournal of Machine Learning Research
Volume14
StatePublished - Nov 1 2013
Externally publishedYes

Keywords

  • Kappa coefficient
  • Penalized regression
  • Selection consistency
  • Stability
  • Tuning

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Statistics and Probability
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Consistent selection of tuning parameters via variable selection stability'. Together they form a unique fingerprint.

  • Cite this