A survey of tuning parameter selection for high-dimensional regression

Yunan Wu, Lan Wang

Research output: Contribution to journalReview articlepeer-review

6 Scopus citations

Abstract

Penalized (or regularized) regression, as represented by lasso and its variants, has become a standard technique for analyzing high-dimensional data when the number of variables substantially exceeds the sample size. The performance of penalized regression relies crucially on the choice of the tuning parameter, which determines the amount of regularization and hence the sparsity level of the fitted model. The optimal choice of tuning parameter depends on both the structure of the design matrix and the unknown random error distribution (variance, tail behavior, etc.). This article reviews the current literature of tuning parameter selection for high-dimensional regression from both the theoretical and practical perspectives. We discuss various strategies that choose the tuning parameter to achieve prediction accuracy or support recovery. We also review several recently proposed methods for tuning-free high-dimensional regression.

Original languageEnglish (US)
Pages (from-to)209-226
Number of pages18
JournalAnnual Review of Statistics and Its Application
Volume7
DOIs
StatePublished - Mar 7 2020
Externally publishedYes

Keywords

  • Bayesian information criterion
  • BIC
  • cross-validation
  • lasso
  • scaled lasso
  • square-root lasso
  • tuning parameter

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint

Dive into the research topics of 'A survey of tuning parameter selection for high-dimensional regression'. Together they form a unique fingerprint.

Cite this