Applications of Hybrid Monte Carlo to Bayesian Generalized Linear Models: Quasicomplete Separation and Neural Networks

Research output: Contribution to journalArticle

23 Scopus citations

Abstract

The "leapfrog" hybrid Monte Carlo algorithm is a simple and effective MCMC method for fitting Bayesian generalized linear models with canonical link. The algorithm leads to large trajectories over the posterior and a rapidly mixing Markov chain, having superior performance over conventional methods in difficult problems like logistic regression with quasicomplete separation. This method offers a very attractive solution to this common problem, providing a method for identifying datasets that are quasicomplete separated, and for identifying the covariates that are at the root of the problem. The method is also quite successful in fitting generalized linear models in which the link function is extended to include a feedforward neural network. With a large number of hidden units, however, or when the dataset becomes large, the computations required in calculating the gradient in each trajectory can become very demanding. In this case, it is best to mix the algorithm with multivariate random walk Metropolis-Hastings. However, this entails very little additional programming work.

Original languageEnglish (US)
Pages (from-to)779-799
Number of pages21
JournalJournal of Computational and Graphical Statistics
Volume8
Issue number4
DOIs
StatePublished - Dec 1999
Externally publishedYes

Keywords

  • Bayesian hierarchical models
  • Feedforward neural networks
  • Leapfrog algorithm
  • Markov chain Monte Carlo
  • Random walk Metropolis-Hastings

ASJC Scopus subject areas

  • Mathematics(all)
  • Computational Mathematics
  • Statistics and Probability

Fingerprint Dive into the research topics of 'Applications of Hybrid Monte Carlo to Bayesian Generalized Linear Models: Quasicomplete Separation and Neural Networks'. Together they form a unique fingerprint.

  • Cite this