Improper versus finitely additive distributions as limits of countably additive probabilities

The Bayesian paradigm with proper priors can be extended either to improper distributions or to finitely additive probabilities (FAPs). Improper distributions and diffuse FAPs can be seen as limits of proper distribution sequences for specific convergence modes. In this paper, we compare these two kinds of limits. We show that improper distributions and FAPs represent two distinct features of the limit behavior of a sequence of proper distribution. More specifically, an improper distribution characterizes the behavior of the sequence inside the domain, whereas diffuse FAPs characterizes how the mass concentrates on the boundary of the domain. Therefore, a diffuse FAP cannot be seen as the counterpart of an improper distribution. As an illustration, we consider several approach to define uniform FAP distributions on natural numbers as an equivalent of improper flat prior. We also show that expected logarithmic convergence may depend on the chosen sequence of compact sets.

     

Functional Compatibility,Markov Chains,and Gibbs Sampling with Improper Posteriors

Abstract

The members of a set of conditional probability density functions are called compatible if there exists a joint probability density function that generates them. We generalize this concept by calling the conditionals functionally compatible if there exists a non-negative function that behaves like a joint density as far as generating the conditionals according to the probability calculus, but whose integral over the whole space is not necessarily finite. A necessary and sufficient condition for functional compatibility is given that provides a method of calculating this function, if it exists. A Markov transition function is then constructed using a set of functionally compatible conditional densities and it is shown, using the compatibility results, that the associated Markov chain is positive recurrent if and only if the conditionals are compatible. A Gibbs Markov chain, constructed via “Gibbs conditionals” from a hierarchical model with an improper posterior, is a special case. Therefore, the results of this article can be used to evaluate the consequences of applying the Gibbs sampler when the posterior's impropriety is unknown to the user. Our results cannot, however, be used to detect improper posteriors. Monte Carlo approximations based on Gibbs chains are shown to have undesirable limiting behavior when the posterior is improper. The results are applied to a Bayesian hierarchical one-way random effects model with an improper posterior distribution. The model is simple, but also quite similar to some models with improper posteriors that have been used in conjunction with the Gibbs sampler in the literature.     

Bayesian inference using a noninformative prior for linear Gaussian random coefficient regression with inhomogeneous within-class variances

A Bayesian inference for a linear Gaussian random coefficient regression model with inhomogeneous within-class variances is presented. The model is motivated by an application in metrology, but it may well find interest in other fields. We consider the selection of a noninformative prior for the Bayesian inference to address applications where the available prior knowledge is either vague or shall be ignored. The noninformative prior is derived by applying the Berger and Bernardo reference prior principle with the means of the random coefficients forming the parameters of interest. We show that the resulting posterior is proper and specify conditions for the existence of first and second moments of the marginal posterior. Simulation results are presented which suggest good frequentist properties of the proposed inference. The calibration of sonic nozzle data is considered as an application from metrology. The proposed inference is applied to these data and the results are compared to those obtained by alternative approaches.     

Triply fuzzy function approximation for hierarchical Bayesian inference

We prove that three independent fuzzy systems can uniformly approximate Bayesian posterior probability density functions by approximating the prior and likelihood probability densities as well as the hyperprior probability densities that underly the priors. This triply fuzzy function approximation extends the recent theorem for uniformly approximating the posterior density by approximating just the prior and likelihood densities. This approximation allows users to state priors and hyper-priors in words or rules as well as to adapt them from sample data. A fuzzy system with just two rules can exactly represent common closed-form probability densities so long as they are bounded. The function approximators can also be neural networks or any other type of uniform function approximator. Iterative fuzzy Bayesian inference can lead to rule explosion. We prove that conjugacy in the if-part set functions for prior, hyperprior, and likelihood fuzzy approximators reduces rule explosion. We also prove that a type of semi-conjugacy of if-part set functions for those fuzzy approximators results in fewer parameters in the fuzzy posterior approximator.     

A bayesian approach to the probability density estimation

Summary A Bayesian procedure for the probability density estimation is proposed. The procedure is based on the multinomial logit transformations of the parameters of a finely segmented histogram model. The smoothness of the estimated density is guaranteed by the introduction of a prior distribution of the parameters. The estimates of the parameters are defined as the mode of the posterior distribution. The prior distribution has several adjustable parameters (hyper-parameters), whose values are chosen so that ABIC (Akaike's Bayesian Information Criterion) is minimized. The basic procedure is developed under the assumption that the density is defined on a bounded interval. The handling of the general case where the support of the density function is not necessarily bounded is also discussed. The practical usefulness of the procedure is demonstrated by numerical examples. The Institute of Statistical Mathematics     

Agreeing to disagree with lexicographic prior beliefs

The robustness of Aumann’s seminal agreement theorem with respect to the common prior assumption is considered. More precisely, we show by means of an example that two Bayesian agents with almost identical prior beliefs can agree to completely disagree on their posterior beliefs. Besides, a more detailed agent model is introduced where posterior beliefs are formed on the basis of lexicographic prior beliefs. We then generalize Aumann’s agreement theorem to lexicographic prior beliefs and show that only a slight perturbation of the common lexicographic prior assumption at some–even arbitrarily deep–level is already compatible with common knowledge of completely opposed posterior beliefs. Hence, agents can actually agree to disagree even if there is only a slight deviation from the common prior assumption.     

An admissibility proof using an adaptive sequence of smoother proper priors approaching the target improper prior

A sufficient condition for the admissibility of generalized Bayes estimators of the location vector of spherically symmetric distribution under squared error loss is derived. This is as strong a condition as that of Brown [L.D. Brown, Admissible estimators, recurrent diffusions, and insoluble boundary value problems, Ann. Math. Statist. 42 (1971) 855–903] under normality. In particular we establish the admissibility of generalized Bayes estimators with respect to the harmonic prior and priors with slightly heavier tails than the harmonic prior. The key to our proof is an adaptive sequence of smooth proper priors approaching an improper prior fast enough to establish admissibility.     

Existence of Bayesian Estimates for the Polychotomous Quantal Response Models

This paper investigates the existence of Bayesian estimates for polychotomous quantal response models using a uniform improper prior distribution on the regression parameters. Necessary and sufficient conditions for the propriety of the posterior distribution with a general link function are established. In addition, the sufficient conditions for the existence of the posterior moments and the posterior moment generating function are obtained. It is also found that the propriety guarantees the existence of the maximum likelihood estimate.     

Priors for Bayesian adaptive spline smoothing

Adaptive smoothing has been proposed for curve-fitting problems where the underlying function is spatially inhomogeneous. Two Bayesian adaptive smoothing models, Bayesian adaptive smoothing splines on a lattice and Bayesian adaptive P-splines, are studied in this paper. Estimation is fully Bayesian and carried out by efficient Gibbs sampling. Choice of prior is critical in any Bayesian non-parametric regression method. We use objective priors on the first level parameters where feasible, specifically independent Jeffreys priors (right Haar priors) on the implied base linear model and error variance, and we derive sufficient conditions on higher level components to ensure that the posterior is proper. Through simulation, we demonstrate that the common practice of approximating improper priors by proper but diffuse priors may lead to invalid inference, and we show how appropriate choices of proper but only weakly informative priors yields satisfactory inference.     

Risks of Using Improper Priors with Gibbs Sampling and Autocorrelated Errors

Abstract

In this article we examine the use of Gibbs sampling to estimate the autocorrelation coefficient in a linear regression model. Researchers had previously experienced difficulty with moderate-to-high positive autocorrelated errors; estimates could be unstable and sometimes failed to converge. We show that the cause of this problem is that the use of an improper prior leads to an improper posterior, although the conditionals are proper, and hence a formal Gibbs sampler can be constructed. The problem is solved by the use of a vague but proper prior. In this simple case many of the calculations can be done analytically and it serves as a warning as to the uncritical use of improper priors with Gibbs sampling.     

Posterior consistency for partially observed Markov models

We establish the posterior consistency for parametric, partially observed, fully dominated Markov models. The prior is assumed to assign positive probability to all neighborhoods of the true parameter, for a distance induced by the expected Kullback–Leibler divergence between the parametric family members’ Markov transition densities. This assumption is easily checked in general. In addition, we show that the posterior consistency is implied by the consistency of the maximum likelihood estimator. The result is extended to possibly improper priors and non-stationary observations. Finally, we check our assumptions on a linear Gaussian model and a well-known stochastic volatility model.     

用于指数可靠性增长模型的一类新的先验分布

该文提出了可用于指数分布产品四种可靠性增长试验方案的一类新的先验分布. 这类先验分布以条件分布形式给出, 它适合可靠性增长试验中的各种情况. 各阶段的条件均值和条件方差的表达式被获得, 先验分布的形式与它们的参数间的关系被讨论. 这些结果有助于与专家意见相结合.本文还给出试验末尾产品可靠性的后验密度, Bayesian估计和Bayesian下限.     

Bayesian nonparametric regression with varying residual density

We consider the problem of robust Bayesian inference on the mean regression function allowing the residual density to change flexibly with predictors. The proposed class of models is based on a Gaussian process (GP) prior for the mean regression function and mixtures of Gaussians for the collection of residual densities indexed by predictors. Initially considering the homoscedastic case, we propose priors for the residual density based on probit stick-breaking mixtures. We provide sufficient conditions to ensure strong posterior consistency in estimating the regression function, generalizing existing theory focused on parametric residual distributions. The homoscedastic priors are generalized to allow residual densities to change nonparametrically with predictors through incorporating GP in the stick-breaking components. This leads to a robust Bayesian regression procedure that automatically down-weights outliers and influential observations in a locally adaptive manner. The methods are illustrated using simulated and real data applications.     

Multiperiod Bayesian forecasts forAR models

The multiperiod Bayesian forecast under the normal-gamma prior assumption for univariateAR models with strongly exogenous variables is investigated. A two-stage approximate method is proposed to provide an estimator of the posterior predictive density for any future observation in a convenient closed form. Some properties of the proposed method are proven analytically for a one-step ahead forecast. The precision of the proposed method is examined by using some simulated data and two sets of real data up to lead-twelve-ahead forecasts by comparison with a path sampling method. It is found that most of the results for the two discussed methods are rather close for short period forecast. Especially when sample size is sufficiently large, the estimated predictive density provided by the two-stage method asymptotically converges to the true density. A heuristic proof of this asymptotic property is also presented.     

Fluid dynamic limits of kinetic equations II convergence proofs for the boltzmann equation

Using relative entropy estimates about an absolute Maxwellian, it is shown that any properly scaled sequence of DiPerna-Lions renormalized solutions of some classical Boltzmann equations has fluctuations that converge to an infinitesimal Maxwellian with fluid variables that satisfy the incompressibility and Boussinesq relations. Moreover, if the initial fluctuations entropically converge to an infinitesimal Maxwellian then the limiting fluid variables satisfy a version of the Leray energy inequality. If the sequence satisfies a local momentum conservation assumption, the momentum densities globaly converge to a solution of the Stokes equation. A similar discrete time version of this result holds for the Navier-Stokes limit with an additional mild weak compactness assumption. The continuous time Navier-Stokes limit is also discussed. © 1993 John Wiley & Sons, Inc.     

Ignorance prior distribution of a hyperparameter and Stein's estimator

Summary The concept of ignorance prior distribution is extended to the case of a hyperparameter. This leads to a procedure of formulating the partial ignorance of the original parameter. Its application to the estimation of the mean of a multivariate normal distribution with a particular hyperparameterized prior distribution of the mean leads to an improper prior distribution with the corresponding posterior mean very close to the James-Stein estimate. The Institute of Statistical Mathematics     

位置参数分布族中Fiducial分布与后验分布的关系

本文考虑本质位置参数分布族中,参数的Fiducial分布与后验分布的等同问题．首先讨论了如何给出Fiducial分布,分析结果表明以分布函数形式给出Fiducial分布要比密度函数形式合理,同时,证明了所给的Fiducial分布具有频率性质．然后,研究在参数受到单侧限制时,Fiducial分布与后验分布等同的问题,给出的充要条件是分布族为指数分布族,此时,先验分布是一个广义先验分布,它不能被Lebesgue测度控制．最后,证明了在参数限制在一个有限区间内时,Fiducial分布与任何先验(包括广义先验分布)下的后验分布不等同．     

Asymptotic Expansions of Posterior Distributions in Nonregular Cases

We study the asymptotic behaviour of the posterior distributions for a one-parameter family of discontinuous densities. It is shown that a suitably centered and normalized posterior converges almost surely to an exponential limit in the total variation norm. Further, asymptotic expansions for the density, distribution function, moments and quantiles of the posterior are also obtained. It is to be noted that, in view of the results of Ghosh et al. (1994, Statistical Decision Theory and Related Topics V, 183-199, Springer, New York) and Ghosal et al. (1995, Ann. Statist., 23, 2145-2152), the nonregular cases considered here are essentially the only ones for which the posterior distributions converge. The results obtained here are also supported by a simulation experiment.     

Admissibility and minimaxity of generalized Bayes estimators for spherically symmetric family

We give a sufficient condition for admissibility of generalized Bayes estimators of the location vector of spherically symmetric distribution under squared error loss. Compared to the known results for the multivariate normal case, our sufficient condition is very tight and is close to being a necessary condition. In particular, we establish the admissibility of generalized Bayes estimators with respect to the harmonic prior and priors with slightly heavier tail than the harmonic prior. We use the theory of regularly varying functions to construct a sequence of smooth proper priors approaching an improper prior fast enough for establishing the admissibility. We also discuss conditions of minimaxity of the generalized Bayes estimator with respect to the harmonic prior.     

Robust Bayesian Inference for Seemingly Unrelated Regressions with Elliptical Errors

Bayesian inference is considered for the seemingly unrelated regressions with an elliptically contoured error distribution. We show that the posterior distribution of the regression parameters and the predictive distribution of future observations under elliptical errors assumption are identical to those obtained under independently distributed normal errors when an improper prior is used. This gives inference robustness with respect to departures from the reference case of independent sampling from the normal distribution.