Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models

Hemant Ishwaran, Mahmoud Zarepour

Research output: Contribution to journalArticle

144 Citations (Scopus)

Abstract

We present some easy-to-construct random probability measures which approximate the Dirichlet process and an extension which we will call the beta two-parameter process. The nature of these constructions makes it simple to implement Markov chain Monte Carlo algorithms for fitting nonparametric hierarchical models and mixtures of non-parametric hierarchical models. For the Dirichlet process, we consider a truncation approximation as well as a weak limit approximation based on a mixture of Dirichlet processes. The same type of truncation approximation can also be applied to the beta two-parameter process. Both methods lead to posteriors which can be fitted using Markov chain Monte Carlo algorithms that take advantage of blocked coordinate updates. These algorithms promote rapid mixing of the Markov chain and can be readily applied to normal mean mixture models and to density estimation problems. We prefer the truncation approximations, since a simple device for monitoring the adequacy of the approximation can be easily computed from the output of the Gibbs sampler. Furthermore, for the Dirichlet process, the truncation approximation offers an exponentially higher degree of accuracy over the weak limit approximation for the same computational effort. We also find that a certain beta two-parameter process may be suitable for finite mixture modelling because the distinct number of sampled values from this process tends to match closely the number of components of the underlying mixture distribution.

Original languageEnglish
Pages (from-to)371-390
Number of pages20
JournalBiometrika
Volume87
Issue number2
StatePublished - Dec 1 2000
Externally publishedYes

Fingerprint

Markov Chains
Hierarchical Model
Markov Chain Monte Carlo
Markov processes
Process Model
Dirichlet
Two Parameters
Truncation
Dirichlet Process
Approximation
Weak Limit
Markov Chain Monte Carlo Algorithms
Nonparametric Model
Random Probability Measure
Equipment and Supplies
Mixture of Dirichlet Processes
samplers
Mixture Modeling
Mixture Distribution
Finite Mixture

Keywords

  • Almost sure truncation
  • Generalised Dirichlet distribution
  • Mixture of Dirichlet processes
  • Nonparametric hierarchical model
  • Normal mean mixture
  • Random probability measure
  • Weak convergence in distribution

ASJC Scopus subject areas

  • Agricultural and Biological Sciences(all)
  • Agricultural and Biological Sciences (miscellaneous)
  • Statistics and Probability
  • Mathematics(all)
  • Applied Mathematics

Cite this

Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models. / Ishwaran, Hemant; Zarepour, Mahmoud.

In: Biometrika, Vol. 87, No. 2, 01.12.2000, p. 371-390.

Research output: Contribution to journalArticle

@article{d2a897be799d4118ae25ab1bb942537e,
title = "Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models",
abstract = "We present some easy-to-construct random probability measures which approximate the Dirichlet process and an extension which we will call the beta two-parameter process. The nature of these constructions makes it simple to implement Markov chain Monte Carlo algorithms for fitting nonparametric hierarchical models and mixtures of non-parametric hierarchical models. For the Dirichlet process, we consider a truncation approximation as well as a weak limit approximation based on a mixture of Dirichlet processes. The same type of truncation approximation can also be applied to the beta two-parameter process. Both methods lead to posteriors which can be fitted using Markov chain Monte Carlo algorithms that take advantage of blocked coordinate updates. These algorithms promote rapid mixing of the Markov chain and can be readily applied to normal mean mixture models and to density estimation problems. We prefer the truncation approximations, since a simple device for monitoring the adequacy of the approximation can be easily computed from the output of the Gibbs sampler. Furthermore, for the Dirichlet process, the truncation approximation offers an exponentially higher degree of accuracy over the weak limit approximation for the same computational effort. We also find that a certain beta two-parameter process may be suitable for finite mixture modelling because the distinct number of sampled values from this process tends to match closely the number of components of the underlying mixture distribution.",
keywords = "Almost sure truncation, Generalised Dirichlet distribution, Mixture of Dirichlet processes, Nonparametric hierarchical model, Normal mean mixture, Random probability measure, Weak convergence in distribution",
author = "Hemant Ishwaran and Mahmoud Zarepour",
year = "2000",
month = "12",
day = "1",
language = "English",
volume = "87",
pages = "371--390",
journal = "Biometrika",
issn = "0006-3444",
publisher = "Oxford University Press",
number = "2",

}

TY - JOUR

T1 - Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models

AU - Ishwaran, Hemant

AU - Zarepour, Mahmoud

PY - 2000/12/1

Y1 - 2000/12/1

N2 - We present some easy-to-construct random probability measures which approximate the Dirichlet process and an extension which we will call the beta two-parameter process. The nature of these constructions makes it simple to implement Markov chain Monte Carlo algorithms for fitting nonparametric hierarchical models and mixtures of non-parametric hierarchical models. For the Dirichlet process, we consider a truncation approximation as well as a weak limit approximation based on a mixture of Dirichlet processes. The same type of truncation approximation can also be applied to the beta two-parameter process. Both methods lead to posteriors which can be fitted using Markov chain Monte Carlo algorithms that take advantage of blocked coordinate updates. These algorithms promote rapid mixing of the Markov chain and can be readily applied to normal mean mixture models and to density estimation problems. We prefer the truncation approximations, since a simple device for monitoring the adequacy of the approximation can be easily computed from the output of the Gibbs sampler. Furthermore, for the Dirichlet process, the truncation approximation offers an exponentially higher degree of accuracy over the weak limit approximation for the same computational effort. We also find that a certain beta two-parameter process may be suitable for finite mixture modelling because the distinct number of sampled values from this process tends to match closely the number of components of the underlying mixture distribution.

AB - We present some easy-to-construct random probability measures which approximate the Dirichlet process and an extension which we will call the beta two-parameter process. The nature of these constructions makes it simple to implement Markov chain Monte Carlo algorithms for fitting nonparametric hierarchical models and mixtures of non-parametric hierarchical models. For the Dirichlet process, we consider a truncation approximation as well as a weak limit approximation based on a mixture of Dirichlet processes. The same type of truncation approximation can also be applied to the beta two-parameter process. Both methods lead to posteriors which can be fitted using Markov chain Monte Carlo algorithms that take advantage of blocked coordinate updates. These algorithms promote rapid mixing of the Markov chain and can be readily applied to normal mean mixture models and to density estimation problems. We prefer the truncation approximations, since a simple device for monitoring the adequacy of the approximation can be easily computed from the output of the Gibbs sampler. Furthermore, for the Dirichlet process, the truncation approximation offers an exponentially higher degree of accuracy over the weak limit approximation for the same computational effort. We also find that a certain beta two-parameter process may be suitable for finite mixture modelling because the distinct number of sampled values from this process tends to match closely the number of components of the underlying mixture distribution.

KW - Almost sure truncation

KW - Generalised Dirichlet distribution

KW - Mixture of Dirichlet processes

KW - Nonparametric hierarchical model

KW - Normal mean mixture

KW - Random probability measure

KW - Weak convergence in distribution

UR - http://www.scopus.com/inward/record.url?scp=0001677650&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0001677650&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:0001677650

VL - 87

SP - 371

EP - 390

JO - Biometrika

JF - Biometrika

SN - 0006-3444

IS - 2

ER -