A Predictive Approach to Tail Probability Estimation

One of the issues contributing to the success of any extreme value modeling is the choice of the number of upper order statistics used for inference, or equivalently, the selection of an appropriate threshold. In this paper we propose a Bayesian predictive approach to the peaks over threshold method with the purpose of estimating extreme quantiles beyond the range of the data. In the peaks over threshold (POT) method, we assume that the threshold identifies a model with a specified prior probability, from a set of possible models. For each model, the predictive distribution of a future excess over the corresponding threshold is computed, as well as a conditional estimate for the corresponding tail probability. The unconditional tail probability for a given future extreme observation from the unknown distribution is then obtained as an average of the conditional tail estimates with weights given by the posterior probability of each model.     

Dynamically Rescaled Hamiltonian Monte Carlo for Bayesian Hierarchical Models

Dynamically rescaled Hamiltonian Monte Carlo is introduced as a computationally fast and easily implemented method for performing full Bayesian analysis in hierarchical statistical models. The method relies on introducing a modified parameterization so that the reparameterized target distribution has close to constant scaling properties, and thus is easily sampled using standard (Euclidian metric) Hamiltonian Monte Carlo. Provided that the parameterizations of the conditional distributions specifying the hierarchical model are “constant information parameterizations” (CIPs), the relation between the modified- and original parameterization is bijective, explicitly computed, and admit exploitation of sparsity in the numerical linear algebra involved. CIPs for a large catalogue of statistical models are presented, and from the catalogue, it is clear that many CIPs are currently routinely used in statistical computing. A relation between the proposed methodology and a class of explicitly integrated Riemann manifold Hamiltonian Monte Carlo methods is discussed. The methodology is illustrated on several example models, including a model for inflation rates with multiple levels of nonlinearly dependent latent variables. Supplementary materials for this article are available online.     

Applications of Hybrid Monte Carlo to Bayesian Generalized Linear Models: Quasicomplete Separation and Neural Networks

Abstract

The “leapfrog” hybrid Monte Carlo algorithm is a simple and effective MCMC method for fitting Bayesian generalized linear models with canonical link. The algorithm leads to large trajectories over the posterior and a rapidly mixing Markov chain, having superior performance over conventional methods in difficult problems like logistic regression with quasicomplete separation. This method offers a very attractive solution to this common problem, providing a method for identifying datasets that are quasicomplete separated, and for identifying the covariates that are at the root of the problem. The method is also quite successful in fitting generalized linear models in which the link function is extended to include a feedforward neural network. With a large number of hidden units, however, or when the dataset becomes large, the computations required in calculating the gradient in each trajectory can become very demanding. In this case, it is best to mix the algorithm with multivariate random walk Metropolis—Hastings. However, this entails very little additional programming work.     

An Autoregressive Ordered Probit Model With Application to High-Frequency Financial Data

This article introduces a model that can be considered as an autoregressive extension of the ordered probit model. For parameter estimation we first develop a standard Gibbs sampler which however exhibits bad convergence properties. Using a special transformation group on the sample space we develop a grouped move multigrid Monte Carlo (GM-MGMC) Gibbs sampler and illustrate its fundamental superiority in convergence compared to the standard sampler. To be able to compare the autoregressive ordered probit (AOP) model to other models we further provide an estimation procedure for the marginal likelihood which enables us to compute Bayes factors. We apply the new model to absolute price changes of the IBM stock traded on December 4, 2000, at the New York Stock Exchange. To detect whether the data contain an autoregressive structure we then fit the AOP model as well as the common ordered probit (OP) model to the data. By estimating the corresponding Bayes factor we show that the AOP model fits the data decisively better than the common OP model.     

Properties of Prior and Posterior Distributions for Multivariate Categorical Response Data Models

In this article, we model multivariate categorical (binary and ordinal) response data using a very rich class of scale mixture of multivariate normal (SMMVN) link functions to accommodate heavy tailed distributions. We consider both noninformative as well as informative prior distributions for SMMVN-link models. The notation of informative prior elicitation is based on available similar historical studies. The main objectives of this article are (i) to derive theoretical properties of noninformative and informative priors as well as the resulting posteriors and (ii) to develop an efficient Markov chain Monte Carlo algorithm to sample from the resulting posterior distribution. A real data example from prostate cancer studies is used to illustrate the proposed methodologies.     

A dynamic programming approach to efficient sampling from Boltzmann distributions

Markov chain methods for Boltzmann sampling work in phases with decreasing temperatures. The number of transitions in each phase crucially affects terminal state distribution. We employ dynamic programming to allocate iterations to phases to improve guarantees on sample quality. Numerical experiments on the Ising model are presented.     

Fast and Accurate Approximation of the Full Conditional for Gamma Shape Parameters

The gamma distribution arises frequently in Bayesian models, but there is not an easy-to-use conjugate prior for the shape parameter of a gamma. This inconvenience is usually dealt with by using either Metropolis–Hastings moves, rejection sampling methods, or numerical integration. However, in models with a large number of shape parameters, these existing methods are slower or more complicated than one would like, making them burdensome in practice. It turns out that the full conditional distribution of the gamma shape parameter is well approximated by a gamma distribution, even for small sample sizes, when the prior on the shape parameter is also a gamma distribution. This article introduces a quick and easy algorithm for finding a gamma distribution that approximates the full conditional distribution of the shape parameter. We empirically demonstrate the speed and accuracy of the approximation across a wide range of conditions. If exactness is required, the approximation can be used as a proposal distribution for Metropolis–Hastings. Supplementary material for this article is available online.     

Markov Chain Monte Carlo Convergence Assessment via Two-Way Analysis of Variance

Abstract

In this article we discuss the problem of assessing the performance of Markov chain Monte Carlo (MCMC) algorithms on the basis of simulation output. In essence, we extend the original ideas of Gelman and Rubin and, more recently, Brooks and Gelman, to problems where we are able to split the variation inherent within the MCMC simulation output into two distinct groups. We show how such a diagnostic may be useful in assessing the performance of MCMC samplers addressing model choice problems, such as the reversible jump MCMC algorithm. In the model choice context, we show how the reversible jump MCMC simulation output for parameters that retain a coherent interpretation throughout the simulation, can be used to assess convergence. By considering various decompositions of the sampling variance of this parameter, we can assess the performance of our MCMC sampler in terms of its mixing properties both within and between models and we illustrate our approach in both the graphical Gaussian models and normal mixtures context. Finally, we provide an example of the application of our diagnostic to the assessment of the influence of different starting values on MCMC simulation output, thereby illustrating the wider utility of our method beyond the Bayesian model choice and reversible jump MCMC context.     

Reparameterized and Marginalized Posterior and Predictive Sampling for Complex Bayesian Geostatistical Models

This article proposes a four-pronged approach to efficient Bayesian estimation and prediction for complex Bayesian hierarchical Gaussian models for spatial and spatiotemporal data. The method involves reparameterizing the covariance structure of the model, reformulating the means structure, marginalizing the joint posterior distribution, and applying a simplex-based slice sampling algorithm. The approach permits fusion of point-source data and areal data measured at different resolutions and accommodates nonspatial correlation and variance heterogeneity as well as spatial and/or temporal correlation. The method produces Markov chain Monte Carlo samplers with low autocorrelation in the output, so that fewer iterations are needed for Bayesian inference than would be the case with other sampling algorithms. Supplemental materials are available online.     

运用时空模型厘定湖北省县级水稻产量保险的纯费率

本文建立了23个综合考虑水稻单产的时间效应、空间效应和时空交互效应的时空模型。利用湖北省1991-2007年县级水稻单产数据,借助WinBUGS进行Gibbs抽样估计,根据DIC准则进行选择最优模型,预测1992-2009年各县市的水稻单产。基于2008年预测单产及其后验分布,厘定县级水稻产量保险的纯费率。研究的主要结论:1992-2007年的预测值与实际值比较接近,且预测的水稻单产的标准差都比较小,表明所选择模型具有良好的短期预测能力;相邻地域的纯费率比较接近;所厘定的纯费率与蒙特卡洛(Monte Carlo,MC)误差正相关,Pearson相关系数为0.6793,MC误差包含来自时间、空间以及两者相互作用带来的不确定性。     

Functional Compatibility,Markov Chains,and Gibbs Sampling with Improper Posteriors

Abstract

The members of a set of conditional probability density functions are called compatible if there exists a joint probability density function that generates them. We generalize this concept by calling the conditionals functionally compatible if there exists a non-negative function that behaves like a joint density as far as generating the conditionals according to the probability calculus, but whose integral over the whole space is not necessarily finite. A necessary and sufficient condition for functional compatibility is given that provides a method of calculating this function, if it exists. A Markov transition function is then constructed using a set of functionally compatible conditional densities and it is shown, using the compatibility results, that the associated Markov chain is positive recurrent if and only if the conditionals are compatible. A Gibbs Markov chain, constructed via “Gibbs conditionals” from a hierarchical model with an improper posterior, is a special case. Therefore, the results of this article can be used to evaluate the consequences of applying the Gibbs sampler when the posterior's impropriety is unknown to the user. Our results cannot, however, be used to detect improper posteriors. Monte Carlo approximations based on Gibbs chains are shown to have undesirable limiting behavior when the posterior is improper. The results are applied to a Bayesian hierarchical one-way random effects model with an improper posterior distribution. The model is simple, but also quite similar to some models with improper posteriors that have been used in conjunction with the Gibbs sampler in the literature.     

E-Bayesian estimation for the Lomax distribution based on type-II censored data

This paper is concerned with using the E-Bayesian method for computing estimates of the unknown parameter and some survival time parameters e.g. reliability and hazard functions of Lomax distribution based on type-II censored data. These estimates are derived based on a conjugate prior for the parameter under the balanced squared error loss function. A comparison between the new method and the corresponding Bayes and maximum likelihood techniques is conducted using the Monte Carlo simulation.     

Nonparametric Bayesian topic modelling with the hierarchical Pitman–Yor processes

The Dirichlet process and its extension, the Pitman–Yor process, are stochastic processes that take probability distributions as a parameter. These processes can be stacked up to form a hierarchical nonparametric Bayesian model. In this article, we present efficient methods for the use of these processes in this hierarchical context, and apply them to latent variable models for text analytics. In particular, we propose a general framework for designing these Bayesian models, which are called topic models in the computer science community. We then propose a specific nonparametric Bayesian topic model for modelling text from social media. We focus on tweets (posts on Twitter) in this article due to their ease of access. We find that our nonparametric model performs better than existing parametric models in both goodness of fit and real world applications.     

A Bayesian inference approach to identify a Robin coefficient in one-dimensional parabolic problems

This paper investigates a nonlinear inverse problem associated with the heat conduction problem of identifying a Robin coefficient from boundary temperature measurement. A Bayesian inference approach is presented for the solution of this problem. The prior modeling is achieved via the Markov random field (MRF). The use of a hierarchical Bayesian method for automatic selection of the regularization parameter in the function estimation inverse problem is discussed. The Markov chain Monte Carlo (MCMC) algorithm is used to explore the posterior state space. Numerical results indicate that MRF provides an effective prior regularization, and the Bayesian inference approach can provide accurate estimates as well as uncertainty quantification to the solution of the inverse problem.     

Spatially Adaptive Bayesian Penalized Regression Splines (P-splines)

In this article we study penalized regression splines (P-splines), which are low-order basis splines with a penalty to avoid undersmoothing. Such P-splines are typically not spatially adaptive, and hence can have trouble when functions are varying rapidly. Our approach is to model the penalty parameter inherent in the P-spline method as a heteroscedastic regression function. We develop a full Bayesian hierarchical structure to do this and use Markov chain Monte Carlo techniques for drawing random samples from the posterior for inference. The advantage of using a Bayesian approach to P-splines is that it allows for simultaneous estimation of the smooth functions and the underlying penalty curve in addition to providing uncertainty intervals of the estimated curve. The Bayesian credible intervals obtained for the estimated curve are shown to have pointwise coverage probabilities close to nominal. The method is extended to additive models with simultaneous spline-based penalty functions for the unknown functions. In simulations, the approach achieves very competitive performance with the current best frequentist P-spline method in terms of frequentist mean squared error and coverage probabilities of the credible intervals, and performs better than some of the other Bayesian methods.     

Bayesian Inference on a Mixture Model With Spatial Dependence

We introduce a new technique to select the number of components of a mixture model with spatial dependence. The method consists of an estimation of the integrated completed likelihood based on a Laplace’s approximation and a new technique to deal with the normalizing constant intractability of the hidden Potts model. Our proposal is applied to a real satellite image. Supplementary materials are available online.     

A Practical Sequential Stopping Rule for High-Dimensional Markov Chain Monte Carlo

A current challenge for many Bayesian analyses is determining when to terminate high-dimensional Markov chain Monte Carlo simulations. To this end, we propose using an automated sequential stopping procedure that terminates the simulation when the computational uncertainty is small relative to the posterior uncertainty. Further, we show this stopping rule is equivalent to stopping when the effective sample size is sufficiently large. Such a stopping rule has previously been shown to work well in settings with posteriors of moderate dimension. In this article, we illustrate its utility in high-dimensional simulations while overcoming some current computational issues. As examples, we consider two complex Bayesian analyses on spatially and temporally correlated datasets. The first involves a dynamic space-time model on weather station data and the second a spatial variable selection model on fMRI brain imaging data. Our results show the sequential stopping rule is easy to implement, provides uncertainty estimates, and performs well in high-dimensional settings. Supplementary materials for this article are available online.     

��Ҷ˹���Ϸ�λ�ع��Gibbs�����㷨

Most regression modeling is based on traditional mean regression which results in non-robust estimation results for non-normal errors. Compared to conventional mean regression, composite quantile regression (CQR) may produce more robust parameters estimation. Based on a composite asymmetric Laplace distribution (CALD), we build a Bayesian hierarchical model for the weighted CQR (WCQR). The Gibbs sampler algorithm of Bayesian WCQR is developed to implement posterior inference. Finally, the proposed method are illustrated by some simulation studies and a real data analysis.     

Semiparametric Modeling and Estimation of Instrumental Variable Models

We apply Bayesian methods to a model involving a binary nonrandom treatment intake variable and an instrumental variable in which the functional forms of some of the covariates in both the treatment intake and outcome distributions are unknown. Continuous and binary response variables are considered. Under the assumption that the functional form is additive in the covariates, we develop efficient Markov chain Monte Carlo-based approaches for summarizing the posterior distribution and for comparing various alternative models via marginal likelihoods and Bayes factors. We show in a simulation experiment that the methods are capable of recovering the unknown functions and are sensitive neither to the sample size nor to the degree of confounding as measured by the correlation between the errors in the treatment and response equations. In the binary response case, however, estimation of the average treatment effect requires larger sample sizes, especially when the degree of confounding is high. The methods are applied to an example dealing with the effect on wages of more than 12 years of education.