Consistency of spike and slab regression

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Spike and slab models are a popular and attractive variable selection approach in regression settings. Applications for these models have blossomed over the last decade and they are increasingly being used in challenging problems. At the same time, theory for spike and slab models has not kept pace with the applications. There are many gaps in what we know about their theoretical properties. An important property known to hold in these models is selective shrinkage: a unique property whereby the posterior mean is shrunk toward zero for non-informative variables only. This property has been shown to hold under orthogonality for continuous priors under the modified class of rescaled spike and slab models. In this paper, we extend this result to the general case and prove an oracle property for the posterior mean under a discrete two-component prior. An immediate consequence is that a strong selective shrinkage property holds. Interestingly, the conditions needed for our result to hold in the non-orthogonal setting are more stringent than in the orthogonal case and amount to a type of enforced sparsity condition that must be met by the prior.

Original languageEnglish
Pages (from-to)1920-1928
Number of pages9
JournalStatistics and Probability Letters
Volume81
Issue number12
DOIs
StatePublished - Dec 1 2011

Fingerprint

Spike
Regression
Posterior Mean
Shrinkage
Oracle Property
Model
Variable Selection
Orthogonality
Sparsity
Zero

Keywords

  • Oracle property
  • Posterior mean
  • Rescaling
  • Shrinkage
  • Two-component prior

ASJC Scopus subject areas

  • Statistics, Probability and Uncertainty
  • Statistics and Probability

Cite this

Consistency of spike and slab regression. / Ishwaran, Hemant; Rao, Jonnagadda S.

In: Statistics and Probability Letters, Vol. 81, No. 12, 01.12.2011, p. 1920-1928.

Research output: Contribution to journalArticle

@article{fde415e9f85b4f7da515f3075ba85811,
title = "Consistency of spike and slab regression",
abstract = "Spike and slab models are a popular and attractive variable selection approach in regression settings. Applications for these models have blossomed over the last decade and they are increasingly being used in challenging problems. At the same time, theory for spike and slab models has not kept pace with the applications. There are many gaps in what we know about their theoretical properties. An important property known to hold in these models is selective shrinkage: a unique property whereby the posterior mean is shrunk toward zero for non-informative variables only. This property has been shown to hold under orthogonality for continuous priors under the modified class of rescaled spike and slab models. In this paper, we extend this result to the general case and prove an oracle property for the posterior mean under a discrete two-component prior. An immediate consequence is that a strong selective shrinkage property holds. Interestingly, the conditions needed for our result to hold in the non-orthogonal setting are more stringent than in the orthogonal case and amount to a type of enforced sparsity condition that must be met by the prior.",
keywords = "Oracle property, Posterior mean, Rescaling, Shrinkage, Two-component prior",
author = "Hemant Ishwaran and Rao, {Jonnagadda S}",
year = "2011",
month = "12",
day = "1",
doi = "10.1016/j.spl.2011.08.005",
language = "English",
volume = "81",
pages = "1920--1928",
journal = "Statistics and Probability Letters",
issn = "0167-7152",
publisher = "Elsevier",
number = "12",

}

TY - JOUR

T1 - Consistency of spike and slab regression

AU - Ishwaran, Hemant

AU - Rao, Jonnagadda S

PY - 2011/12/1

Y1 - 2011/12/1

N2 - Spike and slab models are a popular and attractive variable selection approach in regression settings. Applications for these models have blossomed over the last decade and they are increasingly being used in challenging problems. At the same time, theory for spike and slab models has not kept pace with the applications. There are many gaps in what we know about their theoretical properties. An important property known to hold in these models is selective shrinkage: a unique property whereby the posterior mean is shrunk toward zero for non-informative variables only. This property has been shown to hold under orthogonality for continuous priors under the modified class of rescaled spike and slab models. In this paper, we extend this result to the general case and prove an oracle property for the posterior mean under a discrete two-component prior. An immediate consequence is that a strong selective shrinkage property holds. Interestingly, the conditions needed for our result to hold in the non-orthogonal setting are more stringent than in the orthogonal case and amount to a type of enforced sparsity condition that must be met by the prior.

AB - Spike and slab models are a popular and attractive variable selection approach in regression settings. Applications for these models have blossomed over the last decade and they are increasingly being used in challenging problems. At the same time, theory for spike and slab models has not kept pace with the applications. There are many gaps in what we know about their theoretical properties. An important property known to hold in these models is selective shrinkage: a unique property whereby the posterior mean is shrunk toward zero for non-informative variables only. This property has been shown to hold under orthogonality for continuous priors under the modified class of rescaled spike and slab models. In this paper, we extend this result to the general case and prove an oracle property for the posterior mean under a discrete two-component prior. An immediate consequence is that a strong selective shrinkage property holds. Interestingly, the conditions needed for our result to hold in the non-orthogonal setting are more stringent than in the orthogonal case and amount to a type of enforced sparsity condition that must be met by the prior.

KW - Oracle property

KW - Posterior mean

KW - Rescaling

KW - Shrinkage

KW - Two-component prior

UR - http://www.scopus.com/inward/record.url?scp=80052530414&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80052530414&partnerID=8YFLogxK

U2 - 10.1016/j.spl.2011.08.005

DO - 10.1016/j.spl.2011.08.005

M3 - Article

VL - 81

SP - 1920

EP - 1928

JO - Statistics and Probability Letters

JF - Statistics and Probability Letters

SN - 0167-7152

IS - 12

ER -