Gibbs Sampling Methods for Stick-Breaking Priors

Hemant Ishwaran, Lancelot F. James

Research output: Contribution to journalArticle

770 Citations (Scopus)

Abstract

A rich and flexible class of random probability measures, which we call stick-breaking priors, can be constructed using a sequence of independent beta random variables. Examples of random measures that have this characterization include the Dirichlet process, its two-parameter extension, the two-parameter Poisson-Dirichlel process, finite dimensional Dirichlet priors, and beta two-parameter processes. The rich nature of stick-breaking priors offers Bayesians a useful class of priors for nonparametric problems, while the similar construction used in each prior can be exploited to develop a general computational procedure for fitting them. In this article we present two general types of Gibbs samplers that can be used to lit posteriors of Bayesiun hierarchical models based on stick-breaking priors. The first type of Gibbs sampler, referred to as a Pólya urn Gibbs sampler, is a generalized version of a widely used Gibbs sampling method currently employed for Dirichlet process computing. This method applies to stick-breaking priors with a known Pólya urn churacterization, that is, priors with an explicit and simple prediction rule. Our second method, the blocked Gibbs sampler, is based on an entirely different approach that works by directly sampling values from the posterior of the random measure. The blocked Gibbs sampler can be viewed as a more general approach because it works without requiring an explicit prediction rule. We find that the blocked Gibbs avoids some of the limitations seen with the Pólya urn approach and should be simpler for nonexperts to use.

Original languageEnglish
Pages (from-to)161-173
Number of pages13
JournalJournal of the American Statistical Association
Volume96
Issue number453
StatePublished - Mar 1 2001
Externally publishedYes

Fingerprint

Gibbs Sampler
Gibbs Sampling
Sampling Methods
Two Parameters
Random Measure
Dirichlet Process
Random Probability Measure
Dirichlet Prior
Prediction
Hierarchical Model
Process Parameters
Poisson process
Random variable
Gibbs sampler
Gibbs sampling
Sampling methods
Model-based
Computing

Keywords

  • Blocked Gibbs sampler
  • Dirichlet process
  • Generalized Dirichlel distribution
  • Pitman-Yor process
  • Pólya urn Gibbs sampler
  • Prediction rule
  • Random probability measure
  • Random weights
  • Stable law

ASJC Scopus subject areas

  • Mathematics(all)
  • Statistics and Probability

Cite this

Gibbs Sampling Methods for Stick-Breaking Priors. / Ishwaran, Hemant; James, Lancelot F.

In: Journal of the American Statistical Association, Vol. 96, No. 453, 01.03.2001, p. 161-173.

Research output: Contribution to journalArticle

@article{d8409f2fe50e443f99281f9da13988e4,
title = "Gibbs Sampling Methods for Stick-Breaking Priors",
abstract = "A rich and flexible class of random probability measures, which we call stick-breaking priors, can be constructed using a sequence of independent beta random variables. Examples of random measures that have this characterization include the Dirichlet process, its two-parameter extension, the two-parameter Poisson-Dirichlel process, finite dimensional Dirichlet priors, and beta two-parameter processes. The rich nature of stick-breaking priors offers Bayesians a useful class of priors for nonparametric problems, while the similar construction used in each prior can be exploited to develop a general computational procedure for fitting them. In this article we present two general types of Gibbs samplers that can be used to lit posteriors of Bayesiun hierarchical models based on stick-breaking priors. The first type of Gibbs sampler, referred to as a P{\'o}lya urn Gibbs sampler, is a generalized version of a widely used Gibbs sampling method currently employed for Dirichlet process computing. This method applies to stick-breaking priors with a known P{\'o}lya urn churacterization, that is, priors with an explicit and simple prediction rule. Our second method, the blocked Gibbs sampler, is based on an entirely different approach that works by directly sampling values from the posterior of the random measure. The blocked Gibbs sampler can be viewed as a more general approach because it works without requiring an explicit prediction rule. We find that the blocked Gibbs avoids some of the limitations seen with the P{\'o}lya urn approach and should be simpler for nonexperts to use.",
keywords = "Blocked Gibbs sampler, Dirichlet process, Generalized Dirichlel distribution, Pitman-Yor process, P{\'o}lya urn Gibbs sampler, Prediction rule, Random probability measure, Random weights, Stable law",
author = "Hemant Ishwaran and James, {Lancelot F.}",
year = "2001",
month = "3",
day = "1",
language = "English",
volume = "96",
pages = "161--173",
journal = "Journal of the American Statistical Association",
issn = "0162-1459",
publisher = "Taylor and Francis Ltd.",
number = "453",

}

TY - JOUR

T1 - Gibbs Sampling Methods for Stick-Breaking Priors

AU - Ishwaran, Hemant

AU - James, Lancelot F.

PY - 2001/3/1

Y1 - 2001/3/1

N2 - A rich and flexible class of random probability measures, which we call stick-breaking priors, can be constructed using a sequence of independent beta random variables. Examples of random measures that have this characterization include the Dirichlet process, its two-parameter extension, the two-parameter Poisson-Dirichlel process, finite dimensional Dirichlet priors, and beta two-parameter processes. The rich nature of stick-breaking priors offers Bayesians a useful class of priors for nonparametric problems, while the similar construction used in each prior can be exploited to develop a general computational procedure for fitting them. In this article we present two general types of Gibbs samplers that can be used to lit posteriors of Bayesiun hierarchical models based on stick-breaking priors. The first type of Gibbs sampler, referred to as a Pólya urn Gibbs sampler, is a generalized version of a widely used Gibbs sampling method currently employed for Dirichlet process computing. This method applies to stick-breaking priors with a known Pólya urn churacterization, that is, priors with an explicit and simple prediction rule. Our second method, the blocked Gibbs sampler, is based on an entirely different approach that works by directly sampling values from the posterior of the random measure. The blocked Gibbs sampler can be viewed as a more general approach because it works without requiring an explicit prediction rule. We find that the blocked Gibbs avoids some of the limitations seen with the Pólya urn approach and should be simpler for nonexperts to use.

AB - A rich and flexible class of random probability measures, which we call stick-breaking priors, can be constructed using a sequence of independent beta random variables. Examples of random measures that have this characterization include the Dirichlet process, its two-parameter extension, the two-parameter Poisson-Dirichlel process, finite dimensional Dirichlet priors, and beta two-parameter processes. The rich nature of stick-breaking priors offers Bayesians a useful class of priors for nonparametric problems, while the similar construction used in each prior can be exploited to develop a general computational procedure for fitting them. In this article we present two general types of Gibbs samplers that can be used to lit posteriors of Bayesiun hierarchical models based on stick-breaking priors. The first type of Gibbs sampler, referred to as a Pólya urn Gibbs sampler, is a generalized version of a widely used Gibbs sampling method currently employed for Dirichlet process computing. This method applies to stick-breaking priors with a known Pólya urn churacterization, that is, priors with an explicit and simple prediction rule. Our second method, the blocked Gibbs sampler, is based on an entirely different approach that works by directly sampling values from the posterior of the random measure. The blocked Gibbs sampler can be viewed as a more general approach because it works without requiring an explicit prediction rule. We find that the blocked Gibbs avoids some of the limitations seen with the Pólya urn approach and should be simpler for nonexperts to use.

KW - Blocked Gibbs sampler

KW - Dirichlet process

KW - Generalized Dirichlel distribution

KW - Pitman-Yor process

KW - Pólya urn Gibbs sampler

KW - Prediction rule

KW - Random probability measure

KW - Random weights

KW - Stable law

UR - http://www.scopus.com/inward/record.url?scp=1842816362&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=1842816362&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:1842816362

VL - 96

SP - 161

EP - 173

JO - Journal of the American Statistical Association

JF - Journal of the American Statistical Association

SN - 0162-1459

IS - 453

ER -