Antithetic Methods for Gibbs Samplers

In this article we propose a modification to the output from Metropolis-within-Gibbs samplers that can lead to substantial reductions in the variance over standard estimates. The idea is simple: at each time step of the algorithm, introduce an extra sample into the estimate that is negatively correlated with the current sample, the rationale being that this provides a two-sample numerical approximation to a Rao–Blackwellized estimate. As the conditional sampling distribution at each step has already been constructed, the generation of the antithetic sample often requires negligible computational effort. Our method is implementable whenever one subvector of the state can be sampled from its full conditional and the corresponding distribution function may be inverted, or the full conditional has a symmetric density. We demonstrate our approach in the context of logistic regression and hierarchical Poisson models. The data and computer code used in this article are available online.     

Performance of the Gibbs,Hit-and-Run,and Metropolis Samplers

Abstract

We consider the performance of three Monte Carlo Markov-chain samplers—the Gibbs sampler, which cycles through coordinate directions; the Hit-and-Run (H&R) sampler, which randomly moves in any direction; and the Metropolis sampler, which moves with a probability that is a ratio of likelihoods. We obtain several analytical results. We provide a sufficient condition of the geometric convergence on a bounded region S for the H&R sampler. For a general region S, we review the Schervish and Carlin sufficient geometric convergence condition for the Gibbs sampler. We show that for a multivariate normal distribution this Gibbs sufficient condition holds and for a bivariate normal distribution the Gibbs marginal sample paths are each an AR(1) process, and we obtain the standard errors of sample means and sample variances, which we later use to verify empirical Monte Carlo results. We empirically compare the Gibbs and H&R samplers on bivariate normal examples. For zero correlation, the Gibbs sampler provides independent data, resulting in better performance than H&R. As the absolute value of the correlation increases, H&R performance improves, with H&R substantially better for correlations above .9. We also suggest and study methods for choosing the number of replications, for estimating the standard error of point estimators and for reducing point-estimator variance. We suggest using a single long run instead of using multiple iid separate runs. We suggest using overlapping batch statistics (obs) to get the standard errors of estimates; additional empirical results show that obs is accurate. Finally, we review the geometric convergence of the Metropolis algorithm and develop a Metropolisized H&R sampler. This sampler works well for high-dimensional and complicated integrands or Bayesian posterior densities.     

Geometric ergodicity of the Gibbs sampler for the Poisson change-point model

Poisson change-point models have been widely used for modelling inhomogeneous time-series of count data. There are a number of methods available for estimating the parameters in these models using iterative techniques such as MCMC. Many of these techniques share the common problem that there does not seem to be a definitive way of knowing the number of iterations required to obtain sufficient convergence. In this paper, we show that the Gibbs sampler of the Poisson change-point model is geometrically ergodic. Establishing geometric ergodicity is crucial from a practical point of view as it implies the existence of a Markov chain central limit theorem, which can be used to obtain standard error estimates. We prove that the transition kernel is a trace-class operator, which implies geometric ergodicity of the sampler. We then provide a useful application of the sampler to a model for the quarterly driver fatality counts for the state of Victoria, Australia.     

Geometric Ergodicity of Metropolis-Hastings Algorithms for Conditional Simulation in Generalized Linear Mixed Models

Conditional simulation is useful in connection with inference and prediction for a generalized linear mixed model. We consider random walk Metropolis and Langevin-Hastings algorithms for simulating the random effects given the observed data, when the joint distribution of the unobserved random effects is multivariate Gaussian. In particular we study the desirable property of geometric ergodicity, which ensures the validity of central limit theorems for Monte Carlo estimates.     

Functional Compatibility,Markov Chains,and Gibbs Sampling with Improper Posteriors

Abstract

The members of a set of conditional probability density functions are called compatible if there exists a joint probability density function that generates them. We generalize this concept by calling the conditionals functionally compatible if there exists a non-negative function that behaves like a joint density as far as generating the conditionals according to the probability calculus, but whose integral over the whole space is not necessarily finite. A necessary and sufficient condition for functional compatibility is given that provides a method of calculating this function, if it exists. A Markov transition function is then constructed using a set of functionally compatible conditional densities and it is shown, using the compatibility results, that the associated Markov chain is positive recurrent if and only if the conditionals are compatible. A Gibbs Markov chain, constructed via “Gibbs conditionals” from a hierarchical model with an improper posterior, is a special case. Therefore, the results of this article can be used to evaluate the consequences of applying the Gibbs sampler when the posterior's impropriety is unknown to the user. Our results cannot, however, be used to detect improper posteriors. Monte Carlo approximations based on Gibbs chains are shown to have undesirable limiting behavior when the posterior is improper. The results are applied to a Bayesian hierarchical one-way random effects model with an improper posterior distribution. The model is simple, but also quite similar to some models with improper posteriors that have been used in conjunction with the Gibbs sampler in the literature.     

Central Limit Theorem for a Class of Nonhomogeneous Random Walks

A spatially nonhomogeneous random walk _t on the grid =^m X ⁿ is considered. Let _t ⁰ be a random walk homogeneous in time and space, and let _t be obtained from it by changing transition probabilities on the set A= X ⁿ, || < , so that the walk remains homogeneous only with respect to the subgroup ⁿ of the group . It is shown that if >m 2 or the drift is distinct from zero, then the central limit theorem holds for _t.     

Model selection of a switching mechanism for financial time series

The threshold autoregressive model with generalized autoregressive conditionally heteroskedastic (GARCH) specification is a popular nonlinear model that captures the well‐known asymmetric phenomena in financial market data. The switching mechanisms of hysteretic autoregressive GARCH models are different from threshold autoregressive model with GARCH as regime switching may be delayed when the hysteresis variable lies in a hysteresis zone. This paper conducts a Bayesian model comparison among competing models by designing an adaptive Markov chain Monte Carlo sampling scheme. We illustrate the performance of three kinds of criteria by comparing models with fat‐tailed and/or skewed errors: deviance information criteria, Bayesian predictive information, and an asymptotic version of Bayesian predictive information. A simulation study highlights the properties of the three Bayesian criteria and the accuracy as well as their favorable performance as model selection tools. We demonstrate the proposed method in an empirical study of 12 international stock markets, providing evidence to strongly support for both models with skew fat‐tailed innovations. Copyright © 2016 John Wiley & Sons, Ltd.     

Properties of Prior and Posterior Distributions for Multivariate Categorical Response Data Models

In this article, we model multivariate categorical (binary and ordinal) response data using a very rich class of scale mixture of multivariate normal (SMMVN) link functions to accommodate heavy tailed distributions. We consider both noninformative as well as informative prior distributions for SMMVN-link models. The notation of informative prior elicitation is based on available similar historical studies. The main objectives of this article are (i) to derive theoretical properties of noninformative and informative priors as well as the resulting posteriors and (ii) to develop an efficient Markov chain Monte Carlo algorithm to sample from the resulting posterior distribution. A real data example from prostate cancer studies is used to illustrate the proposed methodologies.     

Rate of convergence in the central limit theorem for empirical processes

Let _n and be an empirical process and a generalized Brownian bridge, respectively, indexed by a class of real measurable functions. From the central limit theorem for empirical processes it follows that for allr0. In this paper, assuming the class to be countably determined, under certain conditions we obtain an estimate for some constantC. Vapnik-ervonenkis class and the indicators of lower left orthants provide examples of classes considered here.     

���ЧӦģ���з��������Bayes���Ƽ���������

??The Bayes estimators of variance components are derived underweighted square loss function for the balanced one-way classification random effectsmodel with the assumption that variance component has the conjugate prior distribution.The superiorities of the Bayes estimators for variance components to traditional ANOVAestimators are studied in terms of the mean square error (MSE) criterion. Finally, aremark for main results is given.     

��������ɷ�����������ζ�����Сֵ�ľֲ����޶���

??The local limit theorems for the minimum of a random walk with Markovian increments is given, with using Presman's factorization theory. This result implies the asymptotic behaviour of the survival probability for a critical branching process in Markovian depended random environment.     

Two-sided bounds and leading term for rates of convergence in the multivariate central limit theorem

For a sequence of independent and identically distributed random vectors, upper and lower bounds are obtained for the discrepancy between the probability measure P_n, induced by their normalized sum, and the Normal measure Φ. The upper and lower bounds are of the same order of magnitude. These results may be derived by a “leading term” approach, in which a signed measure Q_n is introduced as a first order approximation to P_n − Φ. The purpose of this paper is to investigate properties of the leading term.     

Bayesian Inference on a Mixture Model With Spatial Dependence

We introduce a new technique to select the number of components of a mixture model with spatial dependence. The method consists of an estimation of the integrated completed likelihood based on a Laplace’s approximation and a new technique to deal with the normalizing constant intractability of the hidden Potts model. Our proposal is applied to a real satellite image. Supplementary materials are available online.     

Central Limit Theorems for Exchangeable Random Variables When Limits Are Scale Mixtures of Normals

Central limit theorems for exchangeable random variables are studied when limits are scale mixtures of normals. First, necessary and sufficient conditions are given under the asymptotic tail probability condition for the mixands:

Cayley树上随机场的马尔可夫逼近与一类小偏差定理

通过引进样本相对熵率作为Cayley树上任意随机场与马尔可夫链场之间的偏差的一种度量, 建立了关于状态序偶频率的一类小偏差定理. 证明中应用了研究马尔可夫链强极限定理的一种新的分析方法.     

Bayesian source detection and parameter estimation of a plume model based on sensor network measurements

We consider a network of sensors that measure the intensities of a complex plume composed of multiple absorption–diffusion source components. We address the problem of estimating the plume parameters, including the spatial and temporal source origins and the parameters of the diffusion model for each source, based on a sequence of sensor measurements. The approach not only leads to multiple‐source detection, but also the characterization and prediction of the combined plume in space and time. The parameter estimation is formulated as a Bayesian inference problem, and the solution is obtained using a Markov chain Monte Carlo algorithm. The approach is applied to a simulation study, which shows that an accurate parameter estimation is achievable. Copyright © 2010 John Wiley & Sons, Ltd.     

Rate of Convergence of a Stochastic Particle System for the Smoluchowski Coagulation Equation

By continuing the probabilistic approach of Deaconu et al. (2001), we derive a stochastic particle approximation for the Smoluchowski coagulation equations. A convergence result for this model is obtained. Under quite stringent hypothesis we obtain a central limit theorem associated with our convergence. In spite of these restrictive technical assumptions, the rate of convergence result is interesting because it is the first obtained in this direction and seems to hold numerically under weaker hypothesis. This result answers a question closely connected to the Open Problem 16 formulated by Aldous (1999).     

Fast and Accurate Approximation of the Full Conditional for Gamma Shape Parameters

The gamma distribution arises frequently in Bayesian models, but there is not an easy-to-use conjugate prior for the shape parameter of a gamma. This inconvenience is usually dealt with by using either Metropolis–Hastings moves, rejection sampling methods, or numerical integration. However, in models with a large number of shape parameters, these existing methods are slower or more complicated than one would like, making them burdensome in practice. It turns out that the full conditional distribution of the gamma shape parameter is well approximated by a gamma distribution, even for small sample sizes, when the prior on the shape parameter is also a gamma distribution. This article introduces a quick and easy algorithm for finding a gamma distribution that approximates the full conditional distribution of the shape parameter. We empirically demonstrate the speed and accuracy of the approximation across a wide range of conditions. If exactness is required, the approximation can be used as a proposal distribution for Metropolis–Hastings. Supplementary material for this article is available online.     

Statistical Inference in a Linear Model for Spatially Located Sensors and Random Input

In the paper we consider a random linear model for observations provided by spatially located sensors measuring signals coming from one source. For this model a set of sufficient and complete statistics are found, and it is shown that the maximum likelihood estimators of unknown parameters (characteristics of the source) are functions of those statistics. The problem of nonnegative estimators of variance components of the model is shortly discussed. Comparisons of the mean squared errors of several estimators are given. Numerical example concerning hunting for defects in solar cells is considered in details.     

Inference for the Number of Topics in the Latent Dirichlet Allocation Model via Bayesian Mixture Modeling

In latent Dirichlet allocation, the number of topics, T, is a hyperparameter of the model that must be specified before one can fit the model. The need to specify T in advance is restrictive. One way of dealing with this problem is to put a prior on T, but unfortunately the distribution on the latent variables of the model is then a mixture of distributions on spaces of different dimensions, and estimating this mixture distribution by Markov chain Monte Carlo is very difficult. We present a variant of the Metropolis–Hastings algorithm that can be used to estimate this mixture distribution, and in particular the posterior distribution of the number of topics. We evaluate our methodology on synthetic data and compare it with procedures that are currently used in the machine learning literature. We also give an illustration on two collections of articles from Wikipedia. Supplemental materials for this article are available online.