Applications of Hybrid Monte Carlo to Bayesian Generalized Linear Models: Quasicomplete Separation and Neural Networks

Abstract

The “leapfrog” hybrid Monte Carlo algorithm is a simple and effective MCMC method for fitting Bayesian generalized linear models with canonical link. The algorithm leads to large trajectories over the posterior and a rapidly mixing Markov chain, having superior performance over conventional methods in difficult problems like logistic regression with quasicomplete separation. This method offers a very attractive solution to this common problem, providing a method for identifying datasets that are quasicomplete separated, and for identifying the covariates that are at the root of the problem. The method is also quite successful in fitting generalized linear models in which the link function is extended to include a feedforward neural network. With a large number of hidden units, however, or when the dataset becomes large, the computations required in calculating the gradient in each trajectory can become very demanding. In this case, it is best to mix the algorithm with multivariate random walk Metropolis—Hastings. However, this entails very little additional programming work.     

Bayesian Analysis of a Two-State Markov Modulated Poisson Process

Abstract

We postulate observations from a Poisson process whose rate parameter modulates between two values determined by an unobserved Markov chain. The theory switches from continuous to discrete time by considering the intervals between observations as a sequence of dependent random variables. A result from hidden Markov models allows us to sample from the posterior distribution of the model parameters given the observed event times using a Gibbs sampler with only two steps per iteration.     

Bayesian Inference in the Noncentral Student-t Model

This article takes up Bayesian inference in linear models with disturbances from a noncentral Student-t distribution. The distribution is useful when both long tails and asymmetry are features of the data. The distribution can be expressed as a location-scale mixture of normals with inverse weights distributed according to a chi-square distribution. The computations are performed using Gibbs sampling with data augmentation. An empirical application to Standard and Poor's stock returns indicates that posterior odds strongly favor a noncentral Student-t specification over its symmetric counterpart.     

Estimating Expected Information Gains for Experimental Designs With Application to the Random Fatigue-Limit Model

Expected gain in Shannon information is commonly suggested as a Bayesian design evaluation criterion. Because estimating expected information gains is computationally expensive, examples in which they have been successfully used in identifying Bayes optimal designs are both few and typically quite simplistic. This article discusses in general some properties of estimators of expected information gains based on Markov chain Monte Carlo (MCMC) and Laplacian approximations. We then investigate some issues that arise when applying these methods to the problem of experimental design in the (technically nontrivial) random fatigue-limit model of Pascual and Meeker. An example comparing follow-up designs for a laminate panel study is provided.     

Antithetic Methods for Gibbs Samplers

In this article we propose a modification to the output from Metropolis-within-Gibbs samplers that can lead to substantial reductions in the variance over standard estimates. The idea is simple: at each time step of the algorithm, introduce an extra sample into the estimate that is negatively correlated with the current sample, the rationale being that this provides a two-sample numerical approximation to a Rao–Blackwellized estimate. As the conditional sampling distribution at each step has already been constructed, the generation of the antithetic sample often requires negligible computational effort. Our method is implementable whenever one subvector of the state can be sampled from its full conditional and the corresponding distribution function may be inverted, or the full conditional has a symmetric density. We demonstrate our approach in the context of logistic regression and hierarchical Poisson models. The data and computer code used in this article are available online.     

Towards a Faster Implementation of Density Estimation With Logistic Gaussian Process Priors

A novel method is proposed to compute the Bayes estimate for a logistic Gaussian process prior for density estimation. The method gains speed by drawing samples from the posterior of a finite-dimensional surrogate prior, which is obtained by imputation of the underlying Gaussian process. We establish that imputation results in quite accurate computation. Simulation studies show that accuracy and high speed can be combined. This fact, along with known flexibility of the logistic Gaussian priors for modeling smoothness and recent results on their large support, makes these priors and the resulting density estimate very attractive.     

散度偏大计数数据回归模型的变量选择与模型比较

讨论了具有散度偏大特征计数数据的建模与拟合问题.针对导致数据散度偏大的原因和常用的几类候选模型的结构,分别给出了关于嵌套模型的模型与变量同时选择的Bayes方法和关于非嵌套模型的模型检验与比较方法,并在此基础上进一步完善,提出了较为系统完整的模型与变量选择方法.实际例子说明了方法的具体实现过程和有效性.     

Some generalized limit theorems concerning delayed sums of random sequence

For a sequence of arbitrarily dependent m-valued random variables (Xn) n∈N , the generalized strong limit theorem of the delayed average is investigated. In our proof, we improved the method proposed by Liu [6] . As an application, we also studied some limit properties of delayed average for inhomogeneous Markov chains.     

基于网络均衡的电力市场排污权交易政策研究

本文构建了电力供应链网络均衡模型,网络中包含三级决策主体,分别是:发电厂、电力服务商和用电市场.在分析各级决策者行为的基础上,得到了基于变分不等式的均衡条件,研究了针对发电厂的排污权交易政策设计问题.研究发现:排污交易政策实施后,总排污量不会超过给定的排污权总量;发电厂清洁生产能力的差异对排污权交易政策影响显著.     

Brownian localization： A generalized coupling model yielding a nonergodic Langevin equation description

A minimal system-plus-reservoir model yielding a nonergodic Langevin equation is proposed, which originates from the cubic-spectral density of environmental oscillators and momentum-dependent coupling. This model allows ballistic diffusion and classical localization simultaneously, in which the fluctuation-dissipation relation is still satisfied but the Khinchin theorem is broken. The asymptotical equilibrium for a nonergodic system requires the initial thermal equilibrium, however, when the system starts from nonthermal conditions, it does not approach the equilibration even though a nonlinear potential is used to bound the particle, this can be confirmed by the zerotb law of thermodynamics. In the dynamics of Brownian localization, due to the memory damping function inducing a constant term, our results show that the stationary distribution of the system depends on its initial preparation of coordinate rather than momentum. The coupled oscillator chain with a fixed end boundary acts as a heat bath, which has long been used in studies of collinear atom/solid-surface scattering and lattice vibration, we investigate this problem from the viewpoint of nonergodicity.