Sampling Schemes for Bayesian Variable Selection in Generalized Linear Models

Bayesian approaches to prediction and the assessment of predictive uncertainty in generalized linear models are often based on averaging predictions over different models, and this requires methods for accounting for model uncertainty. When there are linear dependencies among potential predictor variables in a generalized linear model, existing Markov chain Monte Carlo algorithms for sampling from the posterior distribution on the model and parameter space in Bayesian variable selection problems may not work well. This article describes a sampling algorithm based on the Swendsen-Wang algorithm for the Ising model, and which works well when the predictors are far from orthogonality. In problems of variable selection for generalized linear models we can index different models by a binary parameter vector, where each binary variable indicates whether or not a given predictor variable is included in the model. The posterior distribution on the model is a distribution on this collection of binary strings, and by thinking of this posterior distribution as a binary spatial field we apply a sampling scheme inspired by the Swendsen-Wang algorithm for the Ising model in order to sample from the model posterior distribution. The algorithm we describe extends a similar algorithm for variable selection problems in linear models. The benefits of the algorithm are demonstrated for both real and simulated data.     

多重线性回归模型的贝叶斯预报分析

多重线性回归模型的贝叶斯预报分析是贝叶斯线性模型理论的重要组成部分。通过模型系统的统计结构，证明了矩阵正态-Wishart分布为模型参数的共轭先验分布；利用贝叶斯定理，根据模型的样本似然函数和参数的先验分布推导了参数的后验分布；然后，从数学上严格推断了模型的预报分布密度函数，证明了模型预报分布为矩阵t分布。研究结果表明：由于参数先验分布的作用，样本的预报分布与其原统计分布有着本质性的差异，前服从矩阵正态分布，而后为矩阵t分布。     

Improved prediction for a multivariate normal distribution with unknown mean and variance

The prediction problem for a multivariate normal distribution is considered where both mean and variance are unknown. When the Kullback–Leibler loss is used, the Bayesian predictive density based on the right invariant prior, which turns out to be a density of a multivariate t-distribution, is the best invariant and minimax predictive density. In this paper, we introduce an improper shrinkage prior and show that the Bayesian predictive density against the shrinkage prior improves upon the best invariant predictive density when the dimension is greater than or equal to three.     

Bayesian Variable Selection for Median Regression

When the data has heavy tail feature or contains outliers, conventional variable selection methods based on penalized least squares or likelihood functions perform poorly. Based on Bayesian inference method, we study the Bayesian variable selection problem for median linear models. The Bayesian estimation method is proposed by using Bayesian model selection theory and Bayesian estimation method through selecting the Spike and Slab prior for regression coefficients, and the effective posterior Gibbs sampling procedure is also given. Extensive numerical simulations and Boston house price data analysis are used to illustrate the effectiveness of the proposed method.     

Bayesian predictive densities based on superharmonic priors for the 2-dimensional Wishart model

Bayesian predictive densities for the 2-dimensional Wishart model are investigated. The performance of predictive densities is evaluated by using the Kullback–Leibler divergence. It is proved that a Bayesian predictive density based on a prior exactly dominates that based on the Jeffreys prior if the prior density satisfies some geometric conditions. An orthogonally invariant prior is introduced and it is shown that the Bayesian predictive density based on the prior is minimax and dominates that based on the right invariant prior with respect to the triangular group.     

Bayesian l0‐regularized least squares

Bayesian l₀‐regularized least squares is a variable selection technique for high‐dimensional predictors. The challenge is optimizing a nonconvex objective function via search over model space consisting of all possible predictor combinations. Spike‐and‐slab (aka Bernoulli‐Gaussian) priors are the gold standard for Bayesian variable selection, with a caveat of computational speed and scalability. Single best replacement (SBR) provides a fast scalable alternative. We provide a link between Bayesian regularization and proximal updating, which provides an equivalence between finding a posterior mode and a posterior mean with a different regularization prior. This allows us to use SBR to find the spike‐and‐slab estimator. To illustrate our methodology, we provide simulation evidence and a real data example on the statistical properties and computational efficiency of SBR versus direct posterior sampling using spike‐and‐slab priors. Finally, we conclude with directions for future research.     

��λ���ع�ı�Ҷ˹����ѡ�񷽷�

??When the data has heavy tail feature or contains outliers, conventional variable selection methods based on penalized least squares or likelihood functions perform poorly. Based on Bayesian inference method, we study the Bayesian variable selection problem for median linear models. The Bayesian estimation method is proposed by using Bayesian model selection theory and Bayesian estimation method through selecting the Spike and Slab prior for regression coefficients, and the effective posterior Gibbs sampling procedure is also given. Extensive numerical simulations and Boston house price data analysis are used to illustrate the effectiveness of the proposed method.     

Bayesian inference for Rayleigh distribution under progressive censored sample

It is often the case that some information is available on the parameter of failure time distributions from previous experiments or analyses of failure time data. The Bayesian approach provides the methodology for incorporation of previous information with the current data. In this paper, given a progressively type II censored sample from a Rayleigh distribution, Bayesian estimators and credible intervals are obtained for the parameter and reliability function. We also derive the Bayes predictive estimator and highest posterior density prediction interval for future observations. Two numerical examples are presented for illustration and some simulation study and comparisons are performed. Copyright © 2006 John Wiley & Sons, Ltd.     

��I�ͻ�Ͻ�β��ָ��——�������ֲ�����ʧЧģ�͵�ͳ�Ʒ���

In this paper, we investigate a competing risks model based on exponentiated Weibull distribution under Type-I progressively hybrid censoring scheme. To estimate the unknown parameters and reliability function, the maximum likelihood estimators and asymptotic confidence intervals are derived. Since Bayesian posterior density functions cannot be given in closed forms, we adopt Markov chain Monte Carlo method to calculate approximate Bayes estimators and highest posterior density credible intervals. To illustrate the estimation methods, a simulation study is carried out with numerical results. It is concluded that the maximum likelihood estimation and Bayesian estimation can be used for statistical inference in competing risks model under Type-I progressively hybrid censoring scheme.     

Bayesian modelling of financial guarantee insurance

In this paper we model the claim process of financial guarantee insurance, and predict the pure premium and the required amount of risk capital. The data used are from the financial guarantee system of the Finnish statutory pension scheme. The losses in financial guarantee insurance may be devastating during an economic depression (i.e., deep recession). This indicates that the economic business cycle, and in particular depressions, must be taken into account in modelling the claim amounts in financial guarantee insurance. A Markov regime-switching model is used to predict the frequency and severity of future depression periods. The claim amounts are predicted using a transfer function model where the predicted growth rate of the real GNP is an explanatory variable. The pure premium and initial risk reserve are evaluated on the basis of the predictive distribution of claim amounts. Bayesian methods are applied throughout the modelling process. For example, estimation is based on posterior simulation with the Gibbs sampler, and model adequacy is assessed by posterior predictive checking. Simulation results show that the required amount of risk capital is high, even though depressions are an infrequent phenomenon.     

Multiperiod Bayesian forecasts forAR models

The multiperiod Bayesian forecast under the normal-gamma prior assumption for univariateAR models with strongly exogenous variables is investigated. A two-stage approximate method is proposed to provide an estimator of the posterior predictive density for any future observation in a convenient closed form. Some properties of the proposed method are proven analytically for a one-step ahead forecast. The precision of the proposed method is examined by using some simulated data and two sets of real data up to lead-twelve-ahead forecasts by comparison with a path sampling method. It is found that most of the results for the two discussed methods are rather close for short period forecast. Especially when sample size is sufficiently large, the estimated predictive density provided by the two-stage method asymptotically converges to the true density. A heuristic proof of this asymptotic property is also presented.     

Bayesian modelling of the time delay between diagnosis and settlement for Critical Illness Insurance using a Burr generalised-linear-type model

We discuss Bayesian modelling of the delay between dates of diagnosis and settlement of claims in Critical Illness Insurance using a Burr distribution. The data are supplied by the UK Continuous Mortality Investigation and relate to claims settled in the years 1999-2005. There are non-recorded dates of diagnosis and settlement and these are included in the analysis as missing values using their posterior predictive distribution and MCMC methodology. The possible factors affecting the delay (age, sex, smoker status, policy type, benefit amount, etc.) are investigated under a Bayesian approach. A 3-parameter Burr generalised-linear-type model is fitted, where the covariates are linked to the mean of the distribution. Variable selection using Bayesian methodology to obtain the best model with different prior distribution setups for the parameters is also applied. In particular, Gibbs variable selection methods are considered, and results are confirmed using exact marginal likelihood findings and related Laplace approximations. For comparison purposes, a lognormal model is also considered.     

Bayesian adaptive Lasso

We propose the Bayesian adaptive Lasso (BaLasso) for variable selection and coefficient estimation in linear regression. The BaLasso is adaptive to the signal level by adopting different shrinkage for different coefficients. Furthermore, we provide a model selection machinery for the BaLasso by assessing the posterior conditional mode estimates, motivated by the hierarchical Bayesian interpretation of the Lasso. Our formulation also permits prediction using a model averaging strategy. We discuss other variants of this new approach and provide a unified framework for variable selection using flexible penalties. Empirical evidence of the attractiveness of the method is demonstrated via extensive simulation studies and data analysis.     

Bayesian Analysis of Abandonment in Call Center Operations

In this paper, we consider the modeling and the inference of abandonment behavior in call centers. We present several time to event modeling strategies, develop Bayesian inference for posterior and predictive analyses, and discuss implications on call center staffing. Different family of distributions, piecewise time to abandonment models, and mixture models are introduced, and their posterior analysis with censored abandonment data is carried out using Markov chain Monte Carlo methods. We illustrate the implementation of the proposed models using real call center data, present additional insights that can be obtained from the Bayesian analysis, and discuss implications for different customer profiles. Copyright © 2012 John Wiley & Sons, Ltd.     

Bayesian Case Influence Measures for Statistical Models With Missing Data

We examine three Bayesian case influence measures including the φ-divergence, Cook’s posterior mode distance, and Cook’s posterior mean distance for identifying a set of influential observations for a variety of statistical models with missing data including models for longitudinal data and latent variable models in the absence/presence of missing data. Since it can be computationally prohibitive to compute these Bayesian case influence measures in models with missing data, we derive simple first-order approximations to the three Bayesian case influence measures by using the Laplace approximation formula and examine the applications of these approximations to the identification of influential sets. All of the computations for the first-order approximations can be easily done using Markov chain Monte Carlo samples from the posterior distribution based on the full data. Simulated data and an AIDS dataset are analyzed to illustrate the methodology. Supplemental materials for the article are available online.     

Bayesian bandwidth estimation for a semi-functional partial linear regression model with unknown error density

In the context of semi-functional partial linear regression model, we study the problem of error density estimation. The unknown error density is approximated by a mixture of Gaussian densities with means being the individual residuals, and variance a constant parameter. This mixture error density has a form of a kernel density estimator of residuals, where the regression function, consisting of parametric and nonparametric components, is estimated by the ordinary least squares and functional Nadaraya–Watson estimators. The estimation accuracy of the ordinary least squares and functional Nadaraya–Watson estimators jointly depends on the same bandwidth parameter. A Bayesian approach is proposed to simultaneously estimate the bandwidths in the kernel-form error density and in the regression function. Under the kernel-form error density, we derive a kernel likelihood and posterior for the bandwidth parameters. For estimating the regression function and error density, a series of simulation studies show that the Bayesian approach yields better accuracy than the benchmark functional cross validation. Illustrated by a spectroscopy data set, we found that the Bayesian approach gives better point forecast accuracy of the regression function than the functional cross validation, and it is capable of producing prediction intervals nonparametrically.     

Adaptive Markov Chain Monte Carlo for Bayesian Variable Selection

We describe adaptive Markov chain Monte Carlo (MCMC) methods for sampling posterior distributions arising from Bayesian variable selection problems. Point-mass mixture priors are commonly used in Bayesian variable selection problems in regression. However, for generalized linear and nonlinear models where the conditional densities cannot be obtained directly, the resulting mixture posterior may be difficult to sample using standard MCMC methods due to multimodality. We introduce an adaptive MCMC scheme that automatically tunes the parameters of a family of mixture proposal distributions during simulation. The resulting chain adapts to sample efficiently from multimodal target distributions. For variable selection problems point-mass components are included in the mixture, and the associated weights adapt to approximate marginal posterior variable inclusion probabilities, while the remaining components approximate the posterior over nonzero values. The resulting sampler transitions efficiently between models, performing parameter estimation and variable selection simultaneously. Ergodicity and convergence are guaranteed by limiting the adaptation based on recent theoretical results. The algorithm is demonstrated on a logistic regression model, a sparse kernel regression, and a random field model from statistical biophysics; in each case the adaptive algorithm dramatically outperforms traditional MH algorithms. Supplementary materials for this article are available online.     

Distribution-free posterior analysis of econometric models

Very often, one needs to perform (classical or Bayesian) inference, when essentially nothing is known about the distribution of the dependent variable given certain covariates. The paper proposes to approximate the unknown distribution by its non-parametric counterpart—a step function—and treat the points of the support and the corresponding density values, as parameters, whose posterior distributions should be determined based on the available data. The paper proposes distributions should be determined based on the available data. The paper proposes Markov chain Monte Carlo methods to perform posterior analysis, and applies the new method to an analysis of stock returns. Copyright © 1999 John Wiley & Sons, Ltd.     

Bayesian prediction based on a class of shrinkage priors for location-scale models

A class of shrinkage priors for multivariate location-scale models is introduced. We consider Bayesian predictive densities for location-scale models and evaluate performance of them using the Kullback–Leibler divergence. We show that Bayesian predictive densities based on priors in the introduced class asymptotically dominate the best invariant predictive density.     

Bayesian demand updating in the lost sales newsvendor problem: A two-moment approximation

We consider Bayesian updating of demand in a lost sales newsvendor model with censored observations. In a lost sales environment, where the arrival process is not recorded, the exact demand is not observed if it exceeds the beginning stock level, resulting in censored observations. Adopting a Bayesian approach for updating the demand distribution, we develop expressions for the exact posteriors starting with conjugate priors, for negative binomial, gamma, Poisson and normal distributions. Having shown that non-informative priors result in degenerate predictive densities except for negative binomial demand, we propose an approximation within the conjugate family by matching the first two moments of the posterior distribution. The conjugacy property of the priors also ensure analytical tractability and ease of computation in successive updates. In our numerical study, we show that the posteriors and the predictive demand distributions obtained exactly and with the approximation are very close to each other, and that the approximation works very well from both probabilistic and operational perspectives in a sequential updating setting as well.