Bayesian Variable Selection for Median Regression

When the data has heavy tail feature or contains outliers, conventional variable selection methods based on penalized least squares or likelihood functions perform poorly. Based on Bayesian inference method, we study the Bayesian variable selection problem for median linear models. The Bayesian estimation method is proposed by using Bayesian model selection theory and Bayesian estimation method through selecting the Spike and Slab prior for regression coefficients, and the effective posterior Gibbs sampling procedure is also given. Extensive numerical simulations and Boston house price data analysis are used to illustrate the effectiveness of the proposed method.     

��λ���ع�ı�Ҷ˹����ѡ�񷽷�

??When the data has heavy tail feature or contains outliers, conventional variable selection methods based on penalized least squares or likelihood functions perform poorly. Based on Bayesian inference method, we study the Bayesian variable selection problem for median linear models. The Bayesian estimation method is proposed by using Bayesian model selection theory and Bayesian estimation method through selecting the Spike and Slab prior for regression coefficients, and the effective posterior Gibbs sampling procedure is also given. Extensive numerical simulations and Boston house price data analysis are used to illustrate the effectiveness of the proposed method.     

稀疏VAR在股票收益率研究的应用

向量自回归模型(VAR)广泛应用在对时间相依的多元时间序列建模中,但在高维数据建模中,自回归的系数膨胀可能导致噪音估计、不稳定的预测、解释上的困难等问题。在实际应用中,序列的真实模型往往具有稀疏性,因此运用稀疏VAR模型对高维时间序列进行建模,不仅可以解决高维数据带来的上述困难,也有利于寻找高维数据内在的真实模型。本文以10家公司的股票收益率为研究对象,采用3种不同的稀疏估计方法,不但分析了股票收益率之间的动态关系,而且通过实证分析展示了稀疏估计的优势。     

非线性再生散度随机效应模型的贝叶斯分析

非线性再生散度随机效应模型是一类非常广泛的统计模型，包括了线性随机效应模型、非线性随机效应模型、广义线性随机效应模型和指数族非线性随机效应模型等．本文研究非线性再生散度随机效应模型的贝叶斯分析．通过视随机效应为缺失数据以及应用结合Gibbs抽样技术和Metropolis-Hastings算法（简称MH算法）的混合算法获得了模型参数与随机效应的同时贝叶斯估计．最后，用一个模拟研究和一个实际例子说明上述算法的可行眭．     

On the Bayesian Mixture Model and Identifiability

This article is concerned with Bayesian mixture models and identifiability issues. There are two sources of unidentifiability: the well-known likelihood invariance under label switching and the perhaps less well-known parameter identifiability problem. When using latent allocation variables determined by the mixture model, these sources of unidentifiability create arbitrary labeling that renders estimation of the model very difficult. We endeavor to tackle these problems by proposing a prior distribution on the allocations, which provides an explicit interpretation for the labeling by removing gaps with high probability. We propose a Markov chain Monte Carlo (MCMC) estimation method and present supporting illustrations.     

基于动态线性模型的中国狭义货币需求及缺口的贝叶斯估计

本文应用动态线性模型研究我国的狭义货币需求,利用贝叶斯吉布斯抽样方法估计模型的参数和方差,获得了潜在货币需求趋势和货币缺口,对我国的货币供给进行分析并获得一些有益结论,对于央行更好地实施货币管理,保持经济平稳发展有积极意义。     

Bayesian image restoration,with two applications in spatial statistics

There has been much recent interest in Bayesian image analysis, including such topics as removal of blur and noise, detection of object boundaries, classification of textures, and reconstruction of two- or three-dimensional scenes from noisy lower-dimensional views. Perhaps the most straightforward task is that of image restoration, though it is often suggested that this is an area of relatively minor practical importance. The present paper argues the contrary, since many problems in the analysis of spatial data can be interpreted as problems of image restoration. Furthermore, the amounts of data involved allow routine use of computer intensive methods, such as the Gibbs sampler, that are not yet practicable for conventional images. Two examples are given, one in archeology, the other in epidemiology. These are preceded by a partial review of pixel-based Bayesian image analysis.An earlier version of this article was presented at the symposium on the Analysis of Statistical Information held in the Institute of Statistical Mathematics, Tokyo during December 5–8, 1989.This research was carried out partly at the University of Durham, U.K., with the support of an award by the Complex Stochastic Systems Initiative of the Science and Engineering Research Council.     

Sampling Schemes for Bayesian Variable Selection in Generalized Linear Models

Bayesian approaches to prediction and the assessment of predictive uncertainty in generalized linear models are often based on averaging predictions over different models, and this requires methods for accounting for model uncertainty. When there are linear dependencies among potential predictor variables in a generalized linear model, existing Markov chain Monte Carlo algorithms for sampling from the posterior distribution on the model and parameter space in Bayesian variable selection problems may not work well. This article describes a sampling algorithm based on the Swendsen-Wang algorithm for the Ising model, and which works well when the predictors are far from orthogonality. In problems of variable selection for generalized linear models we can index different models by a binary parameter vector, where each binary variable indicates whether or not a given predictor variable is included in the model. The posterior distribution on the model is a distribution on this collection of binary strings, and by thinking of this posterior distribution as a binary spatial field we apply a sampling scheme inspired by the Swendsen-Wang algorithm for the Ising model in order to sample from the model posterior distribution. The algorithm we describe extends a similar algorithm for variable selection problems in linear models. The benefits of the algorithm are demonstrated for both real and simulated data.     

Bayesian analysis of constant elasticity of variance models

This paper focuses on the estimation of some models in finance and in particular, in interest rates. We analyse discretized versions of the constant elasticity of variance (CEV) models where the normal law showing up in the usual discretization of the diffusion part is replaced by a range of heavy‐tailed distributions. A further extension of the model is to allow the elasticity of variance to be a parameter itself. This generalized model allows great flexibility in modelling and simplifies the model implementation considerably using the scale mixtures representation. The mixing parameters provide a means to identify possible outliers and protect inference by down‐weighting the distorting effects of these outliers. For parameter estimation, Bayesian approach is adopted and implemented using the software WinBUGS (Bayesian inference using Gibbs sampler). Results from a real data analysis show that an exponential power distribution with a random shape parameter, which is highly leptokurtic compared with the normal distribution, forms the best CEV model for the data. Copyright © 2006 John Wiley & Sons, Ltd.     

隐马尔可夫因子分析模型的半参数贝叶斯分析

因子模型在刻画潜在因素(因子)与观测变量间的影响关系并进而解释多元观测指标(变量)间的相关性方面具有重要作用.在实际应用中,观测数据往往呈现出时序变异多峰,偏态等特性.将经典的因子分析延伸到带有时齐隐马尔可夫模型的动力因子模型,并建立了半参数贝叶斯分析程序.分块GIBBS抽样器用以后验抽样.经验结果展示所建立的统计程序是有效的.     

SLTBL模型的Bayes分析

利用M arkov cha in M on te C arlo技术对可分离的下三角双线性模型进行B ayes分析.由于参数联合后验密度的复杂性,我们导出了所有的条件后验分布,以便利用G ibbs抽样器方法抽取后验密度的样本.特别地,由于从模型的方向向量的后验分布中直接抽样是困难的,我们特别设计了一个M etropolis-H astings算法以解决该难题.我们用仿真的方法验证了所建议方法的有效性,并成功应用于分析实际数据.     

Bayesian l0‐regularized least squares

Bayesian l₀‐regularized least squares is a variable selection technique for high‐dimensional predictors. The challenge is optimizing a nonconvex objective function via search over model space consisting of all possible predictor combinations. Spike‐and‐slab (aka Bernoulli‐Gaussian) priors are the gold standard for Bayesian variable selection, with a caveat of computational speed and scalability. Single best replacement (SBR) provides a fast scalable alternative. We provide a link between Bayesian regularization and proximal updating, which provides an equivalence between finding a posterior mode and a posterior mean with a different regularization prior. This allows us to use SBR to find the spike‐and‐slab estimator. To illustrate our methodology, we provide simulation evidence and a real data example on the statistical properties and computational efficiency of SBR versus direct posterior sampling using spike‐and‐slab priors. Finally, we conclude with directions for future research.     

非线性均值方差模型的贝叶斯数据删除统计诊断

文章在非线性均值方差模型框架下基于K-L距离研究贝叶斯数据删除影响的统计诊断问题,通过应用Gibbs抽样和MH算法估计贝叶斯数据删除影响诊断统计量.随机模拟研究和红鳟鲑鱼数据的数值例子说明该诊断方法的可行性.     

Some theoretical results on the Grouped Variables Lasso

We consider the linear regression model with Gaussian error. We estimate the unknown parameters by a procedure inspired by the Group Lasso estimator introduced in [22]. We show that this estimator satisfies a sparsity inequality, i.e., a bound in terms of the number of non-zero components of the oracle regression vector. We prove that this bound is better, in some cases, than the one achieved by the Lasso and the Dantzig selector.      

Bayesian selection of primary resolution and wavelet basis functions for wavelet regression

This paper considers a Bayesian approach to selecting a primary resolution and wavelet basis functions. Most of papers on wavelet shrinkage have been focused on thresholding of wavelet coefficients, given a primary resolution which is usually determined by the sample size. However, it turns out that a proper primary resolution is much affected by the shape of an unknown function rather than by the sample size. In particular, Bayesian approaches to wavelet series suffer from computational burdens if the chosen primary resolution is too high. A surplus primary resolution may result in a poor estimate. In this paper, we propose a simple Bayesian method to determine a primary resolution and wavelet basis functions independently of the sample size. Results from a simulation study demonstrate the promising empirical properties of the proposed approach.     

泊松逆高斯回归模型的贝叶斯统计推断

本文研究泊松逆高斯回归模型的贝叶斯统计推断.基于应用Gibbs抽样,Metropolis-Hastings算法以及Multiple-Try Metropolis算法等MCMC统计方法计算模型未知参数和潜变量的联合贝叶斯估计,并引入两个拟合优度统计量来评价提出的泊松逆高斯回归模型的合理性.若干模拟研究与一个实证分析说明方...     

Label Switching in Bayesian Mixture Models: Deterministic Relabeling Strategies

Label switching is a well-known problem in the Bayesian analysis of mixture models. On the one hand, it complicates inference, and on the other hand, it has been perceived as a prerequisite to justify Markov chain Monte Carlo (MCMC) convergence. As a result, nonstandard MCMC algorithms that traverse the symmetric copies of the posterior distribution, and possibly genuine modes, have been proposed. To perform component-specific inference, methods to undo the label switching and to recover the interpretation of the components need to be applied. If latent allocations for the design of the MCMC strategy are included, and the sampler has converged, then labels assigned to each component may change from iteration to iteration. However, observations being allocated together must remain similar, and we use this fundamental fact to derive an easy and efficient solution to the label switching problem. We compare our strategy with other relabeling algorithms on univariate and multivariate data examples and demonstrate improvements over alternative strategies. Supplementary materials for this article are available online.     

Bayesian modelling of the time delay between diagnosis and settlement for Critical Illness Insurance using a Burr generalised-linear-type model

We discuss Bayesian modelling of the delay between dates of diagnosis and settlement of claims in Critical Illness Insurance using a Burr distribution. The data are supplied by the UK Continuous Mortality Investigation and relate to claims settled in the years 1999-2005. There are non-recorded dates of diagnosis and settlement and these are included in the analysis as missing values using their posterior predictive distribution and MCMC methodology. The possible factors affecting the delay (age, sex, smoker status, policy type, benefit amount, etc.) are investigated under a Bayesian approach. A 3-parameter Burr generalised-linear-type model is fitted, where the covariates are linked to the mean of the distribution. Variable selection using Bayesian methodology to obtain the best model with different prior distribution setups for the parameters is also applied. In particular, Gibbs variable selection methods are considered, and results are confirmed using exact marginal likelihood findings and related Laplace approximations. For comparison purposes, a lognormal model is also considered.     

Bayesian and likelihood inference from equally weighted mixtures

Equally weighted mixture models are recommended for situations where it is required to draw precise finite sample inferences requiring population parameters, but where the population distribution is not constrained to belong to a simple parametric family. They lead to an alternative procedure to the Laird-DerSimonian maximum likelihood algorithm for unequally weighted mixture models. Their primary purpose lies in the facilitation of exact Bayesian computations via importance sampling. Under very general sampling and prior specifications, exact Bayesian computations can be based upon an application of importance sampling, referred to as Permutable Bayesian Marginalization (PBM). An importance function based upon a truncated multivariatet-distribution is proposed, which refers to a generalization of the maximum likelihood procedure. The estimation of discrete distributions, by binomial mixtures, and inference for survivor distributions, via mixtures of exponential or Weibull distributions, are considered. Equally weighted mixture models are also shown to lead to an alternative Gibbs sampling methodology to the Lavine-West approach.     

Bayesian Variable and Transformation Selection in Linear Regression

This article suggests a method for variable and transformation selection based on posterior probabilities. Our approach allows for consideration of all possible combinations of untransformed and transformed predictors along with transformed and untransformed versions of the response. To transform the predictors in the model, we use a change-point model, or “change-point transformation,” which can yield more interpretable models and transformations than the standard Box–Tidwell approach. We also address the problem of model uncertainty in the selection of models. By averaging over models, we account for the uncertainty inherent in inference based on a single model chosen from the set of models under consideration. We use a Markov chain Monte Carlo model composition (MC³) method which allows us to average over linear regression models when the space of models under consideration is very large. This considers the selection of variables and transformations at the same time. In an example, we show that model averaging improves predictive performance as compared with any single model that might reasonably be selected, both in terms of overall predictive score and of the coverage of prediction intervals. Software to apply the proposed methodology is available via StatLib.