共查询到20条相似文献,搜索用时 562 毫秒
1.
We describe adaptive Markov chain Monte Carlo (MCMC) methods for sampling posterior distributions arising from Bayesian variable selection problems. Point-mass mixture priors are commonly used in Bayesian variable selection problems in regression. However, for generalized linear and nonlinear models where the conditional densities cannot be obtained directly, the resulting mixture posterior may be difficult to sample using standard MCMC methods due to multimodality. We introduce an adaptive MCMC scheme that automatically tunes the parameters of a family of mixture proposal distributions during simulation. The resulting chain adapts to sample efficiently from multimodal target distributions. For variable selection problems point-mass components are included in the mixture, and the associated weights adapt to approximate marginal posterior variable inclusion probabilities, while the remaining components approximate the posterior over nonzero values. The resulting sampler transitions efficiently between models, performing parameter estimation and variable selection simultaneously. Ergodicity and convergence are guaranteed by limiting the adaptation based on recent theoretical results. The algorithm is demonstrated on a logistic regression model, a sparse kernel regression, and a random field model from statistical biophysics; in each case the adaptive algorithm dramatically outperforms traditional MH algorithms. Supplementary materials for this article are available online. 相似文献
2.
《Journal of computational and graphical statistics》2013,22(1):207-227
This article compares three binary Markov random fields (MRFs) which are popular Bayesian priors for spatial smoothing. These are the Ising prior and two priors based on latent Gaussian MRFs. Concern is given to the selection of a suitable Markov chain Monte Carlo (MCMC) sampling scheme for each prior. The properties of the three priors and sampling schemes are investigated in the context of three empirical examples. The first is a simulated dataset, the second involves a confocal fluorescence microscopy dataset, while the third is based on the analysis of functional magnetic resonance imaging (fMRI) data. In the case of the Ising prior, single site and multi-site Swendsen-Wang sampling schemes are both considered. The single site scheme is shown to work consistently well, while it is shown that the Swendsen-Wang algorithm can have convergence problems. The sampling schemes for the priors are extended to generate the smoothing parameters, so that estimation becomes fully automatic. Although this works well, it is found that for highly contiguous images fixing smoothing parameters to very high values can improve results by injecting additional prior information concerning the level of contiguity in the image. The relative properties of the three binary MRFs are investigated, and it is shown how the Ising prior in particular defines sharp edges and encourages clustering. In addition, one of the latent Gaussian MRF priors is shown to be unable to distinguish between higher levels of smoothing. In the context of the fMRI example we also undertake a simulation study. 相似文献
3.
In this article, we propose a new Bayesian variable selection (BVS) approach via the graphical model and the Ising model, which we refer to as the “Bayesian Ising graphical model” (BIGM). The BIGM is developed by showing that the BVS problem based on the linear regression model can be considered as a complete graph and described by an Ising model with random interactions. There are several advantages of our BIGM: it is easy to (i) employ the single-site updating and cluster updating algorithm, both of which are suitable for problems with small sample sizes and a larger number of variables, (ii) extend this approach to nonparametric regression models, and (iii) incorporate graphical prior information. In our BIGM, the interactions are determined by the linear model coefficients, so we systematically study the performance of different scale normal mixture priors for the model coefficients by adopting the global-local shrinkage strategy. Our results indicate that the best prior for the model coefficients in terms of variable selection should place substantial weight on small, nonzero shrinkage. The methods are illustrated with simulated and real data. Supplementary materials for this article are available online. 相似文献
4.
When the data has heavy tail feature or contains outliers, conventional variable selection methods based on penalized least squares or likelihood functions perform poorly. Based on Bayesian inference method, we study the Bayesian variable selection problem for median linear models. The Bayesian estimation method is proposed by using Bayesian model selection theory and Bayesian estimation method through selecting the Spike and Slab prior for regression coefficients, and the effective posterior Gibbs sampling procedure is also given. Extensive numerical simulations and Boston house price data analysis are used to illustrate the effectiveness of the proposed method. 相似文献
5.
In this paper the usage of a stochastic optimization algorithm as a model search tool is proposed for the Bayesian variable selection problem in generalized linear models. Combining aspects of three well known stochastic optimization algorithms, namely, simulated annealing, genetic algorithm and tabu search, a powerful model search algorithm is produced. After choosing suitable priors, the posterior model probability is used as a criterion function for the algorithm; in cases when it is not analytically tractable Laplace approximation is used. The proposed algorithm is illustrated on normal linear and logistic regression models, for simulated and real-life examples, and it is shown that, with a very low computational cost, it achieves improved performance when compared with popular MCMC algorithms, such as the MCMC model composition, as well as with “vanilla” versions of simulated annealing, genetic algorithm and tabu search. 相似文献
6.
Lizhen Xu Radu V. Craiu Lei Sun Andrew D. Paterson 《Journal of computational and graphical statistics》2016,25(2):405-425
Motivated by genetic association studies of pleiotropy, we propose a Bayesian latent variable approach to jointly study multiple outcomes. The models studied here can incorporate both continuous and binary responses, and can account for serial and cluster correlations. We consider Bayesian estimation for the model parameters, and we develop a novel MCMC algorithm that builds upon hierarchical centering and parameter expansion techniques to efficiently sample from the posterior distribution. We evaluate the proposed method via extensive simulations and demonstrate its utility with an application to an association study of various complication outcomes related to Type 1 diabetes. This article has supplementary material online. 相似文献
7.
Lexing Ying 《计算数学(英文版)》2023,41(3):542-550
This note introduces a method for sampling Ising models with mixed boundary conditions. As an application of annealed importance sampling and the Swendsen-Wang algorithm, the method adopts a sequence of intermediate distributions that keeps the temperature fixed but turns on the boundary condition gradually. The numerical results show that the variance of the sample weights is relatively small. 相似文献
8.
Ray-Bing Chen Chi-Hsiang Chu Shinsheng Yuan Ying Nian Wu 《Journal of computational and graphical statistics》2016,25(3):665-683
This article proposes a Bayesian approach for the sparse group selection problem in the regression model. In this problem, the variables are partitioned into different groups. It is assumed that only a small number of groups are active for explaining the response variable, and it is further assumed that within each active group only a small number of variables are active. We adopt a Bayesian hierarchical formulation, where each candidate group is associated with a binary variable indicating whether the group is active or not. Within each group, each candidate variable is also associated with a binary indicator, too. Thus, the sparse group selection problem can be solved by sampling from the posterior distribution of the two layers of indicator variables. We adopt a group-wise Gibbs sampler for posterior sampling. We demonstrate the proposed method by simulation studies as well as real examples. The simulation results show that the proposed method performs better than the sparse group Lasso in terms of selecting the active groups as well as identifying the active variables within the selected groups. Supplementary materials for this article are available online. 相似文献
9.
In this paper, a Bayesian hierarchical model for variable selection and estimation in the context of binary quantile regression is proposed. Existing approaches to variable selection in a binary classification context are sensitive to outliers, heteroskedasticity or other anomalies of the latent response. The method proposed in this study overcomes these problems in an attractive and straightforward way. A Laplace likelihood and Laplace priors for the regression parameters are proposed and estimated with Bayesian Markov Chain Monte Carlo. The resulting model is equivalent to the frequentist lasso procedure. A conceptional result is that by doing so, the binary regression model is moved from a Gaussian to a full Laplacian framework without sacrificing much computational efficiency. In addition, an efficient Gibbs sampler to estimate the model parameters is proposed that is superior to the Metropolis algorithm that is used in previous studies on Bayesian binary quantile regression. Both the simulation studies and the real data analysis indicate that the proposed method performs well in comparison to the other methods. Moreover, as the base model is binary quantile regression, a much more detailed insight in the effects of the covariates is provided by the approach. An implementation of the lasso procedure for binary quantile regression models is available in the R-package bayesQR. 相似文献
10.
??When the data has heavy tail feature or contains outliers, conventional variable selection methods based on penalized least squares or likelihood functions perform poorly. Based on Bayesian inference method, we study the Bayesian variable selection problem for median linear models. The Bayesian estimation method is proposed by using Bayesian model selection theory and Bayesian estimation method through selecting the Spike and Slab prior for regression coefficients, and the effective posterior Gibbs sampling procedure is also given. Extensive numerical simulations and Boston house price data analysis are used to illustrate the effectiveness of the proposed method. 相似文献
11.
Efficient Bayesian Inference for Multivariate Probit Models With Sparse Inverse Correlation Matrices
Aline Talhouk Arnaud Doucet Kevin Murphy 《Journal of computational and graphical statistics》2013,22(3):739-757
We propose a Bayesian approach for inference in the multivariate probit model, taking into account the association structure between binary observations. We model the association through the correlation matrix of the latent Gaussian variables. Conditional independence is imposed by setting some off-diagonal elements of the inverse correlation matrix to zero and this sparsity structure is modeled using a decomposable graphical model. We propose an efficient Markov chain Monte Carlo algorithm relying on a parameter expansion scheme to sample from the resulting posterior distribution. This algorithm updates the correlation matrix within a simple Gibbs sampling framework and allows us to infer the correlation structure from the data, generalizing methods used for inference in decomposable Gaussian graphical models to multivariate binary observations. We demonstrate the performance of this model and of the Markov chain Monte Carlo algorithm on simulated and real datasets. This article has online supplementary materials. 相似文献
12.
Stefanie Kalus Philipp G. Sämann Ludwig Fahrmeir 《Advances in Data Analysis and Classification》2014,8(1):63-83
Functional magnetic resonance imaging (fMRI) is the most popular technique in human brain mapping, with statistical parametric mapping (SPM) as a classical benchmark tool for detecting brain activity. Smith and Fahrmeir (J Am Stat Assoc 102(478):417–431, 2007) proposed a competing method based on a spatial Bayesian variable selection in voxelwise linear regressions, with an Ising prior for latent activation indicators. In this article, we alternatively link activation probabilities to two types of latent Gaussian Markov random fields (GMRFs) via a probit model. Statistical inference in resulting high-dimensional hierarchical models is based on Markov chain Monte Carlo approaches, providing posterior estimates of activation probabilities and enhancing formation of activation clusters. Three algorithms are proposed depending on GMRF type and update scheme. An application to an active acoustic oddball experiment and a simulation study show a substantial increase in sensitivity compared to existing fMRI activation detection methods like classical SPM and the Ising model. 相似文献
13.
Bayesian l0‐regularized least squares is a variable selection technique for high‐dimensional predictors. The challenge is optimizing a nonconvex objective function via search over model space consisting of all possible predictor combinations. Spike‐and‐slab (aka Bernoulli‐Gaussian) priors are the gold standard for Bayesian variable selection, with a caveat of computational speed and scalability. Single best replacement (SBR) provides a fast scalable alternative. We provide a link between Bayesian regularization and proximal updating, which provides an equivalence between finding a posterior mode and a posterior mean with a different regularization prior. This allows us to use SBR to find the spike‐and‐slab estimator. To illustrate our methodology, we provide simulation evidence and a real data example on the statistical properties and computational efficiency of SBR versus direct posterior sampling using spike‐and‐slab priors. Finally, we conclude with directions for future research. 相似文献
14.
Helio S. Migon Alexandra M. Schmidt Romy E. R. Ravines João B. M. Pereira 《Computational Statistics》2013,28(5):2267-2293
A multimove sampling scheme for the state parameters of non-Gaussian and nonlinear dynamic models for univariate time series is proposed. This procedure follows the Bayesian framework, within a Gibbs sampling algorithm with steps of the Metropolis–Hastings algorithm. This sampling scheme combines the conjugate updating approach for generalized dynamic linear models, with the backward sampling of the state parameters used in normal dynamic linear models. A quite extensive Monte Carlo study is conducted in order to compare the results obtained using our proposed method, conjugate updating backward sampling (CUBS), with those obtained using some algorithms previously proposed in the Bayesian literature. We compare the performance of CUBS with other sampling schemes using two real datasets. Then we apply our algorithm in a stochastic volatility model. CUBS significantly reduces the computing time needed to attain convergence of the chains, and is relatively simple to implement. 相似文献
15.
Christian Lavergne Marie-José Martinez Catherine Trottier 《Computational Statistics》2008,23(1):99-109
This paper focuses on model selection in generalized linear mixed models using an information criterion approach. In these
models in general, the response marginal distribution cannot be analytically derived. Thus, for parameter estimation, two
approximations are revisited both leading to iterative model linearizations. We propose simple model selection criteria adapted
from information criteria and based on the linearized model obtained at convergence of the algorithm. The quality of derived
criteria are evaluated through simulations. 相似文献
16.
《Journal of computational and graphical statistics》2013,22(4):880-896
Hierarchical model specifications using latent variables are frequently used to reflect correlation structure in data. Motivated by the structure of a Bayesian multivariate probit model, we demonstrate a parameter-extended Metropolis-Hastings algorithm for sampling from the posterior distribution of a correlation matrix. Our sampling algorithms lead directly to two readily interpretable families of prior distributions for a correlation matrix. The methodology is illustrated through a simulation study and through an application with repeated binary outcomes on individuals from a study of a suicide prevention intervention. 相似文献
17.
Vitaly Schetinin Jonathan E. Fieldsend Derek Partridge Wojtek J. Krzanowski Richard M. Everson Trevor C. Bailey Adolfo Hernandez 《Journal of Mathematical Modelling and Algorithms》2006,5(4):397-416
Multiple Classifier Systems (MCSs) allow evaluation of the uncertainty of classification outcomes that is of crucial importance for safety critical applications. The uncertainty of classification is determined by a trade-off between the amount of data available for training, the classifier diversity and the required performance. The interpretability of MCSs can also give useful information for experts responsible for making reliable classifications. For this reason Decision Trees (DTs) seem to be attractive classification models for experts. The required diversity of MCSs exploiting such classification models can be achieved by using two techniques, the Bayesian model averaging and the randomised DT ensemble. Both techniques have revealed promising results when applied to real-world problems. In this paper we experimentally compare the classification uncertainty of the Bayesian model averaging with a restarting strategy and the randomised DT ensemble on a synthetic dataset and some domain problems commonly used in the machine learning community. To make the Bayesian DT averaging feasible, we use a Markov Chain Monte Carlo technique. The classification uncertainty is evaluated within an Uncertainty Envelope technique dealing with the class posterior distribution and a given confidence probability. Exploring a full posterior distribution, this technique produces realistic estimates which can be easily interpreted in statistical terms. In our experiments we found out that the Bayesian DTs are superior to the randomised DT ensembles within the Uncertainty Envelope technique. 相似文献
18.
Optimal subset selection among a general family of threshold autoregressive moving-average (TARMA) models is considered. The
usual complexity of model/order selection is increased by capturing the uncertainty of unknown threshold levels and an unknown
delay lag. The Monte Carlo method of Bayesian model averaging provides a possible way to overcome such model uncertainty.
Incorporating with the idea of Bayesian model averaging, a modified stochastic search variable selection method is adapted
to consider subset selection in TARMA models, by adding latent indicator variables for all potential model lags as part of
the proposed Markov chain Monte Carlo sampling scheme. Metropolis–Hastings methods are employed to deal with the well-known
difficulty of including moving-average terms in the model and a novel proposal mechanism is designed for this purpose. Bayesian
comparison of two hyper-parameter settings is carried out via a simulation study. The results demonstrate that the modified
method has favourable performance under reasonable sample size and appropriate settings of the necessary hyper-parameters.
Finally, the application to four real datasets illustrates that the proposed method can provide promising and parsimonious
models from more than 16 million possible subsets. 相似文献
19.
A threshold stochastic volatility (SV) model is used for capturing time-varying volatilities and nonlinearity. Two adaptive Markov chain Monte Carlo (MCMC) methods of model selection are designed for the selection of threshold variables for this family of SV models. The first method is the direct estimation which approximates the model posterior probabilities of competing models. Using parallel MCMC sampling to estimate these probabilities, the best threshold variable is selected with the highest posterior model probability. The second method is to use the deviance information criterion to compare among these competing models and select the best one. Simulation results lead us to conclude that for large samples the posterior model probability approximation method can give an accurate approximation of the posterior probability in Bayesian model selection. The method delivers a powerful and sharp model selection tool. An empirical study of five Asian stock markets provides strong support for the threshold variable which is formulated as a weighted average of important variables. 相似文献
20.
An innovative Bayesian sequential censored sampling inspection method and application to test design
This paper proposes an innovative Bayesian sequential censored sampling inspection method to improve the inspection level and reduce the sample size in acceptance test plans for continuous lots. A mathematical model of Bayesian sequential censored sampling is built, where a new inspection parameter is created and two types of risk are modified. As the core of Bayesian risk formulas, a new structure method of the prior distribution is presented by combining the empirical distribution with the uncertainty of the estimation. To improve the fitting accuracy of parameter estimation, an adaptive genetic algorithm is applied and compared with different parameter estimation methods. In the prior distribution, a prior estimator is introduced to design a sampling plan for continuous lots. Then, three types of producer's and consumer's risks are derived and compared. The simulation results indicate that the modified Bayesian sampling method performs well, with the lowest risks and the smallest sample size. Finally, a new sequential censored sampling plan for continuous lots is designed for the accuracy acceptance test of an aircraft. The test results show that compared with the traditional single sampling plan, the sample size is reduced by 66.7%, saving a vast amount of test costs. 相似文献