Spike and slab variable selection: Frequentist and bayesian strategies

Research output: Contribution to journalReview article

367 Scopus citations

Abstract

Variable selection in the linear regression model takes many apparent faces from both frequentist and Bayesian standpoints. In this paper we introduce a variable selection method referred to as a rescaled spike and slab model. We study the importance of prior hierarchical specifications and draw connections to frequentist generalized ridge regression estimation. Specifically, we study the usefulness of continuous bimodal priors to model hypervariance parameters, and the effect scaling has on the posterior mean through its relationship to penalization. Several model selection strategies, some frequentist and some Bayesian in nature, are developed and studied theoretically. We demonstrate the importance of selective shrinkage for effective variable selection in terms of risk misclassification, and show this is achieved using the posterior from a rescaled spike and slab model. We also show how to verify a procedure's ability to reduce model uncertainty in finite samples using a specialized forward selection strategy. Using this tool, we illustrate the effectiveness of rescaled spike and slab models in reducing model uncertainty.

Original languageEnglish (US)
Pages (from-to)730-773
Number of pages44
JournalAnnals of Statistics
Volume33
Issue number2
DOIs
StatePublished - Apr 1 2005
Externally publishedYes

Keywords

  • Generalized ridge regression
  • Hypervariance
  • Model averaging
  • Model uncertainty
  • Ordinary least squares
  • Penalization
  • Rescaling
  • Shrinkage
  • Stochastic variable selection
  • Zcut

ASJC Scopus subject areas

  • Mathematics(all)
  • Statistics and Probability

Fingerprint Dive into the research topics of 'Spike and slab variable selection: Frequentist and bayesian strategies'. Together they form a unique fingerprint.

  • Cite this