Approximate Dirichlet process computing in finite normal mixtures: Smoothing and prior information

Hemant Ishwaran, Lancelot F. James

Research output: Contribution to journalArticle

99 Citations (Scopus)

Abstract

A rich nonparametric analysis of the finite normal mixture model is obtained by working with a precise truncation approximation of the Dirichlet process. Model fitting is carried out by a simple Gibbs sampling algorithm that directly samples the nonparametric posterior. The proposed sampler mixes well, requires no tuning parameters, and involves only draws from simple distributions, including the draw for the mass parameter that controls clustering, and the draw for the variances with the use of a nonconjugate uniform prior. Working directly with the nonparametric prior is conceptually appealing and among other things leads to graphical methods for studying the posterior mixing distribution as well as penalized MLE procedures for deriving point estimates. We discuss methods for automating selection of priors for the mean and variance components to avoid over or undersmoothing the data. We also look at the effectiveness of incorporating prior information in the form of frequentist point estimates.

Original languageEnglish
Pages (from-to)508-532
Number of pages25
JournalJournal of Computational and Graphical Statistics
Volume11
Issue number3
DOIs
StatePublished - Sep 1 2002
Externally publishedYes

Fingerprint

Normal Mixture
Finite Mixture
Dirichlet Process
Prior Information
Smoothing
Point Estimate
Computing
Maximum likelihood estimation
Mixing Distribution
Graphical Methods
Variance Components
Gibbs Sampling
Tuning
Model Fitting
Parameter Tuning
Sampling
Posterior distribution
Mixture Model
Truncation
Control Parameter

Keywords

  • Almost sure truncation
  • Blocked Gibbs sampler
  • Nonparametric hierarchical model
  • Penalized MLE
  • Pólya urn Gibbs sampling
  • Random probability measure

ASJC Scopus subject areas

  • Mathematics(all)
  • Computational Mathematics
  • Statistics and Probability

Cite this

Approximate Dirichlet process computing in finite normal mixtures : Smoothing and prior information. / Ishwaran, Hemant; James, Lancelot F.

In: Journal of Computational and Graphical Statistics, Vol. 11, No. 3, 01.09.2002, p. 508-532.

Research output: Contribution to journalArticle

@article{2da614ca232c4fb787c1413aceaeb85d,
title = "Approximate Dirichlet process computing in finite normal mixtures: Smoothing and prior information",
abstract = "A rich nonparametric analysis of the finite normal mixture model is obtained by working with a precise truncation approximation of the Dirichlet process. Model fitting is carried out by a simple Gibbs sampling algorithm that directly samples the nonparametric posterior. The proposed sampler mixes well, requires no tuning parameters, and involves only draws from simple distributions, including the draw for the mass parameter that controls clustering, and the draw for the variances with the use of a nonconjugate uniform prior. Working directly with the nonparametric prior is conceptually appealing and among other things leads to graphical methods for studying the posterior mixing distribution as well as penalized MLE procedures for deriving point estimates. We discuss methods for automating selection of priors for the mean and variance components to avoid over or undersmoothing the data. We also look at the effectiveness of incorporating prior information in the form of frequentist point estimates.",
keywords = "Almost sure truncation, Blocked Gibbs sampler, Nonparametric hierarchical model, Penalized MLE, P{\'o}lya urn Gibbs sampling, Random probability measure",
author = "Hemant Ishwaran and James, {Lancelot F.}",
year = "2002",
month = "9",
day = "1",
doi = "10.1198/106186002411",
language = "English",
volume = "11",
pages = "508--532",
journal = "Journal of Computational and Graphical Statistics",
issn = "1061-8600",
publisher = "American Statistical Association",
number = "3",

}

TY - JOUR

T1 - Approximate Dirichlet process computing in finite normal mixtures

T2 - Smoothing and prior information

AU - Ishwaran, Hemant

AU - James, Lancelot F.

PY - 2002/9/1

Y1 - 2002/9/1

N2 - A rich nonparametric analysis of the finite normal mixture model is obtained by working with a precise truncation approximation of the Dirichlet process. Model fitting is carried out by a simple Gibbs sampling algorithm that directly samples the nonparametric posterior. The proposed sampler mixes well, requires no tuning parameters, and involves only draws from simple distributions, including the draw for the mass parameter that controls clustering, and the draw for the variances with the use of a nonconjugate uniform prior. Working directly with the nonparametric prior is conceptually appealing and among other things leads to graphical methods for studying the posterior mixing distribution as well as penalized MLE procedures for deriving point estimates. We discuss methods for automating selection of priors for the mean and variance components to avoid over or undersmoothing the data. We also look at the effectiveness of incorporating prior information in the form of frequentist point estimates.

AB - A rich nonparametric analysis of the finite normal mixture model is obtained by working with a precise truncation approximation of the Dirichlet process. Model fitting is carried out by a simple Gibbs sampling algorithm that directly samples the nonparametric posterior. The proposed sampler mixes well, requires no tuning parameters, and involves only draws from simple distributions, including the draw for the mass parameter that controls clustering, and the draw for the variances with the use of a nonconjugate uniform prior. Working directly with the nonparametric prior is conceptually appealing and among other things leads to graphical methods for studying the posterior mixing distribution as well as penalized MLE procedures for deriving point estimates. We discuss methods for automating selection of priors for the mean and variance components to avoid over or undersmoothing the data. We also look at the effectiveness of incorporating prior information in the form of frequentist point estimates.

KW - Almost sure truncation

KW - Blocked Gibbs sampler

KW - Nonparametric hierarchical model

KW - Penalized MLE

KW - Pólya urn Gibbs sampling

KW - Random probability measure

UR - http://www.scopus.com/inward/record.url?scp=0036749869&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0036749869&partnerID=8YFLogxK

U2 - 10.1198/106186002411

DO - 10.1198/106186002411

M3 - Article

AN - SCOPUS:0036749869

VL - 11

SP - 508

EP - 532

JO - Journal of Computational and Graphical Statistics

JF - Journal of Computational and Graphical Statistics

SN - 1061-8600

IS - 3

ER -