Bayesian shrinkage prediction for the regression problem

We consider Bayesian shrinkage predictions for the Normal regression problem under the frequentist Kullback-Leibler risk function.Firstly, we consider the multivariate Normal model with an unknown mean and a known covariance. While the unknown mean is fixed, the covariance of future samples can be different from that of training samples. We show that the Bayesian predictive distribution based on the uniform prior is dominated by that based on a class of priors if the prior distributions for the covariance and future covariance matrices are rotation invariant.Then, we consider a class of priors for the mean parameters depending on the future covariance matrix. With such a prior, we can construct a Bayesian predictive distribution dominating that based on the uniform prior.Lastly, applying this result to the prediction of response variables in the Normal linear regression model, we show that there exists a Bayesian predictive distribution dominating that based on the uniform prior. Minimaxity of these Bayesian predictions follows from these results.     

Bayesian predictive densities based on superharmonic priors for the 2-dimensional Wishart model

Bayesian predictive densities for the 2-dimensional Wishart model are investigated. The performance of predictive densities is evaluated by using the Kullback–Leibler divergence. It is proved that a Bayesian predictive density based on a prior exactly dominates that based on the Jeffreys prior if the prior density satisfies some geometric conditions. An orthogonally invariant prior is introduced and it is shown that the Bayesian predictive density based on the prior is minimax and dominates that based on the right invariant prior with respect to the triangular group.     

Bayesian MAP model selection of chain event graphs

Chain event graphs are graphical models that while retaining most of the structural advantages of Bayesian networks for model interrogation, propagation and learning, more naturally encode asymmetric state spaces and the order in which events happen than Bayesian networks do. In addition, the class of models that can be represented by chain event graphs for a finite set of discrete variables is a strict superset of the class that can be described by Bayesian networks. In this paper we demonstrate how with complete sampling, conjugate closed form model selection based on product Dirichlet priors is possible, and prove that suitable homogeneity assumptions characterise the product Dirichlet prior on this class of models. We demonstrate our techniques using two educational examples.     

Empirical Bayes predictive densities for high-dimensional normal models

This paper addresses the problem of estimating the density of a future outcome from a multivariate normal model. We propose a class of empirical Bayes predictive densities and evaluate their performances under the Kullback–Leibler (KL) divergence. We show that these empirical Bayes predictive densities dominate the Bayesian predictive density under the uniform prior and thus are minimax under some general conditions. We also establish the asymptotic optimality of these empirical Bayes predictive densities in infinite-dimensional parameter spaces through an oracle inequality.     

Bayesian prediction based on a class of shrinkage priors for location-scale models

A class of shrinkage priors for multivariate location-scale models is introduced. We consider Bayesian predictive densities for location-scale models and evaluate performance of them using the Kullback–Leibler divergence. We show that Bayesian predictive densities based on priors in the introduced class asymptotically dominate the best invariant predictive density.     

Estimation, prediction and the Stein phenomenon under divergence loss

We consider two problems: (1) estimate a normal mean under a general divergence loss introduced in [S. Amari, Differential geometry of curved exponential families — curvatures and information loss, Ann. Statist. 10 (1982) 357-387] and [N. Cressie, T.R.C. Read, Multinomial goodness-of-fit tests, J. Roy. Statist. Soc. Ser. B. 46 (1984) 440-464] and (2) find a predictive density of a new observation drawn independently of observations sampled from a normal distribution with the same mean but possibly with a different variance under the same loss. The general divergence loss includes as special cases both the Kullback-Leibler and Bhattacharyya-Hellinger losses. The sample mean, which is a Bayes estimator of the population mean under this loss and the improper uniform prior, is shown to be minimax in any arbitrary dimension. A counterpart of this result for predictive density is also proved in any arbitrary dimension. The admissibility of these rules holds in one dimension, and we conjecture that the result is true in two dimensions as well. However, the general Baranchick [A.J. Baranchick, a family of minimax estimators of the mean of a multivariate normal distribution, Ann. Math. Statist. 41 (1970) 642-645] class of estimators, which includes the James-Stein estimator and the Strawderman [W.E. Strawderman, Proper Bayes minimax estimators of the multivariate normal mean, Ann. Math. Statist. 42 (1971) 385-388] class of estimators, dominates the sample mean in three or higher dimensions for the estimation problem. An analogous class of predictive densities is defined and any member of this class is shown to dominate the predictive density corresponding to a uniform prior in three or higher dimensions. For the prediction problem, in the special case of Kullback-Leibler loss, our results complement to a certain extent some of the recent important work of Komaki [F. Komaki, A shrinkage predictive distribution for multivariate normal observations, Biometrika 88 (2001) 859-864] and George, Liang and Xu [E.I. George, F. Liang, X. Xu, Improved minimax predictive densities under Kullbak-Leibler loss, Ann. Statist. 34 (2006) 78-92]. While our proposed approach produces a general class of predictive densities (not necessarily Bayes, but not excluding Bayes predictors) dominating the predictive density under a uniform prior. We show also that various modifications of the James-Stein estimator continue to dominate the sample mean, and by the duality of estimation and predictive density results which we will show, similar results continue to hold for the prediction problem as well.     

以截尾伽玛分布为先验分布的产品失效率的贝叶斯估计

本文研究了基于泊松分布的产品失效率估计问题.利用贝叶斯统计推断方法,获得了以截尾伽玛分布为先验分布时,产品失效率的贝叶斯估计和相关性质,推广了以伽玛分布为先验分布的贝叶斯估计结果.     

Bayesian reference analysis for Gaussian Markov random fields

Gaussian Markov random fields (GMRF) are important families of distributions for the modeling of spatial data and have been extensively used in different areas of spatial statistics such as disease mapping, image analysis and remote sensing. GMRFs have been used for the modeling of spatial data, both as models for the sampling distribution of the observed data and as models for the prior of latent processes/random effects; we consider mainly the former use of GMRFs. We study a large class of GMRF models that includes several models previously proposed in the literature. An objective Bayesian analysis is presented for the parameters of the above class of GMRFs, where explicit expressions for the Jeffreys (two versions) and reference priors are derived, and for each of these priors results on posterior propriety of the model parameters are established. We describe a simple MCMC algorithm for sampling from the posterior distribution of the model parameters, and study frequentist properties of the Bayesian inferences resulting from the use of these automatic priors. Finally, we illustrate the use of the proposed GMRF model and reference prior for studying the spatial variability of lip cancer cases in the districts of Scotland over the period 1975-1980.     

Robust Bayesian prediction and estimation under a squared log error loss function

Robust Bayesian analysis is concerned with the problem of making decisions about some future observation or an unknown parameter, when the prior distribution belongs to a class Γ instead of being specified exactly. In this paper, the problem of robust Bayesian prediction and estimation under a squared log error loss function is considered. We find the posterior regret Γ-minimax predictor and estimator in a general class of distributions. Furthermore, we construct the conditional Γ-minimax, most stable and least sensitive prediction and estimation in a gamma model. A prequential analysis is carried out by using a simulation study to compare these predictors.     

Extended Bernstein prior via reinforced urn processes

A reinforced urn process, which induces a prior on the space of mixtures of Bernstein distributions is introduced. A nonparametric Bayesian model based on this prior is presented: the elicitation is treated and some connections with Dirichlet mixtures are given. In the last part of the article, an MCMC algorithm to compute the predictive distribution is discussed.     

Sparse Bayesian hierarchical modeling of high-dimensional clustering problems

Clustering is one of the most widely used procedures in the analysis of microarray data, for example with the goal of discovering cancer subtypes based on observed heterogeneity of genetic marks between different tissues. It is well known that in such high-dimensional settings, the existence of many noise variables can overwhelm the few signals embedded in the high-dimensional space. We propose a novel Bayesian approach based on Dirichlet process with a sparsity prior that simultaneous performs variable selection and clustering, and also discover variables that only distinguish a subset of the cluster components. Unlike previous Bayesian formulations, we use Dirichlet process (DP) for both clustering of samples as well as for regularizing the high-dimensional mean/variance structure. To solve the computational challenge brought by this double usage of DP, we propose to make use of a sequential sampling scheme embedded within Markov chain Monte Carlo (MCMC) updates to improve the naive implementation of existing algorithms for DP mixture models. Our method is demonstrated on a simulation study and illustrated with the leukemia gene expression dataset.     

Asymptotics in Bayesian decision theory with applications to global robustness

We provide the rate of convergence of the Bayes action derived from non smooth loss functions involved in Bayesian robustness. Such loss functions are typically not twice differentiable but admit right and left second derivatives. The asymptotic limit of three measures of global robustness is given. These measures are the range of the Bayes actions set associated with a class of loss functions, the maximum regret of using a particular loss when the subjective loss belongs to a given class and the range of the posterior expected loss when the loss ranges over a given class. An application to prior robustness with density ratio classes is provided.     

Improved prediction for a multivariate normal distribution with unknown mean and variance

The prediction problem for a multivariate normal distribution is considered where both mean and variance are unknown. When the Kullback–Leibler loss is used, the Bayesian predictive density based on the right invariant prior, which turns out to be a density of a multivariate t-distribution, is the best invariant and minimax predictive density. In this paper, we introduce an improper shrinkage prior and show that the Bayesian predictive density against the shrinkage prior improves upon the best invariant predictive density when the dimension is greater than or equal to three.     

Equiaffine structures on statistical manifolds and Bayesian statistics

Relations between equiaffine geometry and Bayesian statistics are studied. A prior distribution in Bayesian statistics is regarded as a volume form on a statistical manifold. Applying equiaffine geometry to Bayesian statistics, the relation between alpha-parallel priors and the Jeffreys prior is given. As geometric results, conditions for a statistical submanifold to have an equiaffine structure are also given.     

Vectors of two-parameter Poisson-Dirichlet processes

The definition of vectors of dependent random probability measures is a topic of interest in applications to Bayesian statistics. They represent dependent nonparametric prior distributions that are useful for modelling observables for which specific covariate values are known. In this paper we propose a vector of two-parameter Poisson-Dirichlet processes. It is well-known that each component can be obtained by resorting to a change of measure of a σ-stable process. Thus dependence is achieved by applying a Lévy copula to the marginal intensities. In a two-sample problem, we determine the corresponding partition probability function which turns out to be partially exchangeable. Moreover, we evaluate predictive and posterior distributions.     

基于先验信息的社会碳成本贝叶斯不确定性分析

针对气候变化及经济影响存在的巨大不确定性,研究了气候变化不确定性以及先验信息对社会碳成本的影响.在贝叶斯理论框架下,采用指数分布刻画气候变化的分布特征,假设尾部变化率是一个随机变量,给出其伽玛先验分布,推导了气候变化分布的贝叶斯先验预测分布.并分别基于指数分布以及帕累托先验预测分布计算了社会碳排放成本.模拟分析发现,在未融合先验信息的情况下,由于尾部概率很小,不管是否修正消费与气候变化之间的关系,截尾社会碳成本和未截尾社会碳成本几乎重合.然而,在利用贝叶斯方法融合先验信息的情况下,社会碳成本容易受到先验信息的影响.但是,通过修正消费与气候变化之间的关系后,发现社会碳成本受先验信息的影响比较少.     

Parameter estimation of a bivariate compound Poisson process

In this article, we review the concept of a Lévy copula to describe the dependence structure of a bivariate compound Poisson process. In this first statistical approach we consider a parametric model for the Lévy copula and estimate the parameters of the full dependent model based on a maximum likelihood approach. This approach ensures that the estimated model remains in the class of multivariate compound Poisson processes. A simulation study investigates the small sample behaviour of the MLEs, where we also suggest a new simulation algorithm. Finally, we apply our method to Danish fire insurance data.     

The penalized profile sampler

The penalized profile sampler for semiparametric inference is an extension of the profile sampler method [B.L. Lee, M.R. Kosorok, J.P. Fine, The profile sampler, Journal of the American Statistical Association 100 (2005) 960-969] obtained by profiling a penalized log-likelihood. The idea is to base inference on the posterior distribution obtained by multiplying a profiled penalized log-likelihood by a prior for the parametric component, where the profiling and penalization are applied to the nuisance parameter. Because the prior is not applied to the full likelihood, the method is not strictly Bayesian. A benefit of this approximately Bayesian method is that it circumvents the need to put a prior on the possibly infinite-dimensional nuisance components of the model. We investigate the first and second order frequentist performance of the penalized profile sampler, and demonstrate that the accuracy of the procedure can be adjusted by the size of the assigned smoothing parameter. The theoretical validity of the procedure is illustrated for two examples: a partly linear model with normal error for current status data and a semiparametric logistic regression model. Simulation studies are used to verify the theoretical results.     

A COMPOUND POISSON MODEL FOR LEARNING DISCRETE BAYESIAN NETWORKS

We introduce here the concept of Bayesian networks, in compound Poisson model, which provides a graphical modeling framework that encodes the joint probability distribution for a set of random variables within a directed acyclic graph. We suggest an approach proposal which offers a new mixed implicit estimator. We show that the implicit approach applied in compound Poisson model is very attractive for its ability to understand data and does not require any prior information. A comparative study between learned estimates given by implicit and by standard Bayesian approaches is established. Under some conditions and based on minimal squared error calculations, we show that the mixed implicit estimator is better than the standard Bayesian and the maximum likelihood estimators. We illustrate our approach by considering a simulation study in the context of mobile communication networks.