Bayesian reference analysis for Gaussian Markov random fields

Gaussian Markov random fields (GMRF) are important families of distributions for the modeling of spatial data and have been extensively used in different areas of spatial statistics such as disease mapping, image analysis and remote sensing. GMRFs have been used for the modeling of spatial data, both as models for the sampling distribution of the observed data and as models for the prior of latent processes/random effects; we consider mainly the former use of GMRFs. We study a large class of GMRF models that includes several models previously proposed in the literature. An objective Bayesian analysis is presented for the parameters of the above class of GMRFs, where explicit expressions for the Jeffreys (two versions) and reference priors are derived, and for each of these priors results on posterior propriety of the model parameters are established. We describe a simple MCMC algorithm for sampling from the posterior distribution of the model parameters, and study frequentist properties of the Bayesian inferences resulting from the use of these automatic priors. Finally, we illustrate the use of the proposed GMRF model and reference prior for studying the spatial variability of lip cancer cases in the districts of Scotland over the period 1975-1980.     

Asymptotic Bayesian structure learning using graph supports for Gaussian graphical models

The theory of Gaussian graphical models is a powerful tool for independence analysis between continuous variables. In this framework, various methods have been conceived to infer independence relations from data samples. However, most of them result in stepwise, deterministic, descent algorithms that are inadequate for solving this issue. More recent developments have focused on stochastic procedures, yet they all base their research on strong a priori knowledge and are unable to perform model selection among the set of all possible models. Moreover, convergence of the corresponding algorithms is slow, precluding applications on a large scale. In this paper, we propose a novel Bayesian strategy to deal with structure learning. Relating graphs to their supports, we convert the problem of model selection into that of parameter estimation. Use of non-informative priors and asymptotic results yield a posterior probability for independence graph supports in closed form. Gibbs sampling is then applied to approximate the full joint posterior density. We finally give three examples of structure learning, one from synthetic data, and the two others from real data.     

Robust Gaussian graphical modeling

A new Gaussian graphical modeling that is robustified against possible outliers is proposed. The likelihood function is weighted according to how the observation is deviated, where the deviation of the observation is measured based on its likelihood. Test statistics associated with the robustified estimators are developed. These include statistics for goodness of fit of a model. An outlying score, similar to but more robust than the Mahalanobis distance, is also proposed. The new scores make it easier to identify outlying observations. A Monte Carlo simulation and an analysis of a real data set show that the proposed method works better than ordinary Gaussian graphical modeling and some other robustified multivariate estimators.     

Asymptotics of Bayesian median loss estimation

We establish the consistency, asymptotic normality, and efficiency for estimators derived by minimizing the median of a loss function in a Bayesian context. We contrast this procedure with the behavior of two Frequentist procedures, the least median of squares (LMS) and the least trimmed squares (LTS) estimators, in regression problems. The LMS estimator is the Frequentist version of our estimator, and the LTS estimator approaches a median-based estimator as the trimming approaches 50% on each side. We argue that the Bayesian median-based method is a good tradeoff between the two Frequentist estimators.     

Asymptotics in Bayesian decision theory with applications to global robustness

We provide the rate of convergence of the Bayes action derived from non smooth loss functions involved in Bayesian robustness. Such loss functions are typically not twice differentiable but admit right and left second derivatives. The asymptotic limit of three measures of global robustness is given. These measures are the range of the Bayes actions set associated with a class of loss functions, the maximum regret of using a particular loss when the subjective loss belongs to a given class and the range of the posterior expected loss when the loss ranges over a given class. An application to prior robustness with density ratio classes is provided.     

Sparse Bayesian hierarchical modeling of high-dimensional clustering problems

Clustering is one of the most widely used procedures in the analysis of microarray data, for example with the goal of discovering cancer subtypes based on observed heterogeneity of genetic marks between different tissues. It is well known that in such high-dimensional settings, the existence of many noise variables can overwhelm the few signals embedded in the high-dimensional space. We propose a novel Bayesian approach based on Dirichlet process with a sparsity prior that simultaneous performs variable selection and clustering, and also discover variables that only distinguish a subset of the cluster components. Unlike previous Bayesian formulations, we use Dirichlet process (DP) for both clustering of samples as well as for regularizing the high-dimensional mean/variance structure. To solve the computational challenge brought by this double usage of DP, we propose to make use of a sequential sampling scheme embedded within Markov chain Monte Carlo (MCMC) updates to improve the naive implementation of existing algorithms for DP mixture models. Our method is demonstrated on a simulation study and illustrated with the leukemia gene expression dataset.     

Equiaffine structures on statistical manifolds and Bayesian statistics

Relations between equiaffine geometry and Bayesian statistics are studied. A prior distribution in Bayesian statistics is regarded as a volume form on a statistical manifold. Applying equiaffine geometry to Bayesian statistics, the relation between alpha-parallel priors and the Jeffreys prior is given. As geometric results, conditions for a statistical submanifold to have an equiaffine structure are also given.     

Asymptotic properties of Bayes estimators for Gaussian Itô-processes with noisy observations

The estimation of a real parameter θ in a linear stochastic differential equation of the simple type is investigated, based on noisy, time continuous observations of X_t. Sufficient conditions on the continuous functions β and σ are given such that the (conditionally normal) Bayes estimators of θ satisfy certain error bounds and are strongly consistent.     

A class of proper priors for Bayesian simultaneous prediction of independent Poisson observables

Simultaneous prediction and parameter inference for the independent Poisson observables model are considered. A class of proper prior distributions for Poisson means is introduced. Bayesian predictive densities and estimators based on priors in the introduced class dominate the Bayesian predictive density and estimator based on the Jeffreys prior under Kullback-Leibler loss.     

Constructing priors based on model size for nondecomposable Gaussian graphical models: A simulation based approach

A method for constructing priors is proposed that allows the off-diagonal elements of the concentration matrix of Gaussian data to be zero. The priors have the property that the marginal prior distribution of the number of nonzero off-diagonal elements of the concentration matrix (referred to below as model size) can be specified flexibly. The priors have normalizing constants for each model size, rather than for each model, giving a tractable number of normalizing constants that need to be estimated. The article shows how to estimate the normalizing constants using Markov chain Monte Carlo simulation and supersedes the method of Wong et al. (2003) [24] because it is more accurate and more general. The method is applied to two examples. The first is a mixture of constrained Wisharts. The second is from Wong et al. (2003) [24] and decomposes the concentration matrix into a function of partial correlations and conditional variances using a mixture distribution on the matrix of partial correlations. The approach detects structural zeros in the concentration matrix and estimates the covariance matrix parsimoniously if the concentration matrix is sparse.     

Robust Bayesian prediction and estimation under a squared log error loss function

Robust Bayesian analysis is concerned with the problem of making decisions about some future observation or an unknown parameter, when the prior distribution belongs to a class Γ instead of being specified exactly. In this paper, the problem of robust Bayesian prediction and estimation under a squared log error loss function is considered. We find the posterior regret Γ-minimax predictor and estimator in a general class of distributions. Furthermore, we construct the conditional Γ-minimax, most stable and least sensitive prediction and estimation in a gamma model. A prequential analysis is carried out by using a simulation study to compare these predictors.     

Bayesian shrinkage prediction for the regression problem

We consider Bayesian shrinkage predictions for the Normal regression problem under the frequentist Kullback-Leibler risk function.Firstly, we consider the multivariate Normal model with an unknown mean and a known covariance. While the unknown mean is fixed, the covariance of future samples can be different from that of training samples. We show that the Bayesian predictive distribution based on the uniform prior is dominated by that based on a class of priors if the prior distributions for the covariance and future covariance matrices are rotation invariant.Then, we consider a class of priors for the mean parameters depending on the future covariance matrix. With such a prior, we can construct a Bayesian predictive distribution dominating that based on the uniform prior.Lastly, applying this result to the prediction of response variables in the Normal linear regression model, we show that there exists a Bayesian predictive distribution dominating that based on the uniform prior. Minimaxity of these Bayesian predictions follows from these results.     

On the false discovery proportion convergence under Gaussian equi-correlation

We study the convergence of the false discovery proportion (FDP) of the Benjamini-Hochberg procedure in the Gaussian equi-correlated model, when the correlation ρ_m converges to zero as the hypothesis number m grows to infinity. In this model, the FDP converges to the false discovery rate (FDR) at rate {min(m,1/ρ_m)}^1/2, which is different from the standard convergence rate m^1/2 holding under independence.     

Flows in networks with dynamic ramification nodes

Combining functional analytical and graph theoretical methods, we investigate flow processes as in the papers [15] and [16], but we change the transmission process in the nodes of the network. Instead of conservation of mass, we assume that the velocity of the outgoing flow mass in the vertices is determined by the total incoming flow mass and by the outgoing flow in the other vertices.     

Moderate deviations for quadratic forms in Gaussian stationary processes

Moderate deviations limit theorem is proved for quadratic forms in zero-mean Gaussian stationary processes. Two particular cases are the cumulative periodogram and the kernel spectral density estimator. We also derive the exponential decay of moderate deviation probabilities of goodness-of-fit tests for the spectral density and then discuss intermediate asymptotic efficiencies of tests.     

Bayesian modeling of several covariance matrices and some results on propriety of the posterior for linear regression with correlated and/or heterogeneous errors

We explore simultaneous modeling of several covariance matrices across groups using the spectral (eigenvalue) decomposition and modified Cholesky decomposition. We introduce several models for covariance matrices under different assumptions about the mean structure. We consider ‘dependence’ matrices, which tend to have many parameters, as constant across groups and/or parsimoniously modeled via a regression formulation. For ‘variances’, we consider both unrestricted across groups and more parsimoniously modeled via log-linear models. In all these models, we explore the propriety of the posterior when improper priors are used on the mean and ‘variance’ parameters (and in some cases, on components of the ‘dependence’ matrices). The models examined include several common Bayesian regression models, whose propriety has not been previously explored, as special cases. We propose a simple approach to weaken the assumption of constant dependence matrices in an automated fashion and describe how to compute Bayes factors to test the hypothesis of constant ‘dependence’ across groups. The models are applied to data from two longitudinal clinical studies.     

Bayesian analysis of multivariate t linear mixed models using a combination of IBF and Gibbs samplers

The multivariate linear mixed model (MLMM) has become the most widely used tool for analyzing multi-outcome longitudinal data. Although it offers great flexibility for modeling the between- and within-subject correlation among multi-outcome repeated measures, the underlying normality assumption is vulnerable to potential atypical observations. We present a fully Bayesian approach to the multivariate t linear mixed model (MtLMM), which is a robust extension of MLMM with the random effects and errors jointly distributed as a multivariate t distribution. Owing to the introduction of too many hidden variables in the model, the conventional Markov chain Monte Carlo (MCMC) method may converge painfully slowly and thus fails to provide valid inference. To alleviate this problem, a computationally efficient inverse Bayes formulas (IBF) sampler coupled with the Gibbs scheme, called the IBF-Gibbs sampler, is developed and shown to be effective in drawing samples from the target distributions. The issues related to model determination and Bayesian predictive inference for future values are also investigated. The proposed methodologies are illustrated with a real example from an AIDS clinical trial and a careful simulation study.     

High breakdown estimators for principal components: the projection-pursuit approach revisited

Li and Chen (J. Amer. Statist. Assoc. 80 (1985) 759) proposed a method for principal components using projection-pursuit techniques. In classical principal components one searches for directions with maximal variance, and their approach consists of replacing this variance by a robust scale measure. Li and Chen showed that this estimator is consistent, qualitative robust and inherits the breakdown point of the robust scale estimator. We complete their study by deriving the influence function of the estimators for the eigenvectors, eigenvalues and the associated dispersion matrix. Corresponding Gaussian efficiencies are presented as well. Asymptotic normality of the estimators has been treated in a paper of Cui et al. (Biometrika 90 (2003) 953), complementing the results of this paper. Furthermore, a simple explicit version of the projection-pursuit based estimator is proposed and shown to be fast to compute, orthogonally equivariant, and having the maximal finite-sample breakdown point property. We will illustrate the method with a real data example.     

A robust estimator of multivariate location based on projection

This paper presents an estimator of location vector based on one-dimensional projection of high dimensional data. The properties of the new estimator including consistency ,asymptotic normality and robustness are discussed. It is proved that the estimator is not only stronglyconsistent and asymptotically normal but also with a breakdown point 1/2 and a bounded influence function.     

On Toeplitz type quadratic functionals of stationary Gaussian processes

Summary A central limit theorem for Toeplitz type quadratic functionals of a stationary Gaussian processX(t),t, is proved, generalizing the result of Avram [1] for discrete time processes. The result is applied to the problem of nonparametric estimation of linear functionals of an unknown spectral density function. We give some upper bounds for the minimax mean square risk of the nonparametric estimators, similar to those by Ibragimov and Has'minskii [12] for a probability density function.