Reparameterized and Marginalized Posterior and Predictive Sampling for Complex Bayesian Geostatistical Models

This article proposes a four-pronged approach to efficient Bayesian estimation and prediction for complex Bayesian hierarchical Gaussian models for spatial and spatiotemporal data. The method involves reparameterizing the covariance structure of the model, reformulating the means structure, marginalizing the joint posterior distribution, and applying a simplex-based slice sampling algorithm. The approach permits fusion of point-source data and areal data measured at different resolutions and accommodates nonspatial correlation and variance heterogeneity as well as spatial and/or temporal correlation. The method produces Markov chain Monte Carlo samplers with low autocorrelation in the output, so that fewer iterations are needed for Bayesian inference than would be the case with other sampling algorithms. Supplemental materials are available online.     

A Computationally Efficient Projection-Based Approach for Spatial Generalized Linear Mixed Models

Inference for spatial generalized linear mixed models (SGLMMs) for high-dimensional non-Gaussian spatial data is computationally intensive. The computational challenge is due to the high-dimensional random effects and because Markov chain Monte Carlo (MCMC) algorithms for these models tend to be slow mixing. Moreover, spatial confounding inflates the variance of fixed effect (regression coefficient) estimates. Our approach addresses both the computational and confounding issues by replacing the high-dimensional spatial random effects with a reduced-dimensional representation based on random projections. Standard MCMC algorithms mix well and the reduced-dimensional setting speeds up computations per iteration. We show, via simulated examples, that Bayesian inference for this reduced-dimensional approach works well both in terms of inference as well as prediction; our methods also compare favorably to existing “reduced-rank” approaches. We also apply our methods to two real world data examples, one on bird count data and the other classifying rock types. Supplementary material for this article is available online.     

A Spatial Markov Model for Climate Extremes

Spatial climate data are often presented as summaries of areal regions such as grid cells, either because they are the output of numerical climate models or to facilitate comparison with numerical climate model output. Extreme value analysis can benefit greatly from spatial methods that borrow information across regions. For Gaussian outcomes, a host of methods that respect the areal nature of the data are available, including conditional and simultaneous autoregressive models. However, to our knowledge, there is no such method in the spatial extreme value analysis literature. In this article, we propose a new method for areal extremes that accounts for spatial dependence using latent clustering of neighboring regions. We show that the proposed model has desirable asymptotic dependence properties and leads to relatively simple computation. Applying the proposed method to North American climate data reveals several local and continental-scale changes in the distribution of precipitation and temperature extremes over time. Supplementary material for this article is available online.     

Spatial Lasso With Applications to GIS Model Selection

Geographic information systems (GIS) organize spatial data in multiple two-dimensional arrays called layers. In many applications, a response of interest is observed on a set of sites in the landscape, and it is of interest to build a regression model from the GIS layers to predict the response at unsampled sites. Model selection in this context then consists not only of selecting appropriate layers, but also of choosing appropriate neighborhoods within those layers. We formalize this problem as a linear model and propose the use of Lasso to simultaneously select variables, choose neighborhoods, and estimate parameters. Spatially dependent errors are accounted for using generalized least squares and spatial smoothness in selected coefficients is incorporated through use of a priori spatial covariance structure. This leads to a modification of the Lasso procedure, called spatial Lasso. The spatial Lasso can be implemented by a fast algorithm and it performs well in numerical examples, including an application to prediction of soil moisture. The methodology is also extended to generalized linear models. Supplemental materials including R computer code and data analyzed in this article are available online.     

Online Variational Bayes Inference for High-Dimensional Correlated Data

High-dimensional data with hundreds of thousands of observations are becoming commonplace in many disciplines. The analysis of such data poses many computational challenges, especially when the observations are correlated over time and/or across space. In this article, we propose flexible hierarchical regression models for analyzing such data that accommodate serial and/or spatial correlation. We address the computational challenges involved in fitting these models by adopting an approximate inference framework. We develop an online variational Bayes algorithm that works by incrementally reading the data into memory one portion at a time. The performance of the method is assessed through simulation studies. The methodology is applied to analyze signal intensity in MRI images of subjects with knee osteoarthritis, using data from the Osteoarthritis Initiative. Supplementary materials for this article are available online.     

Zero Expectile Processes and Bayesian Spatial Regression

We introduce new classes of stationary spatial processes with asymmetric, sub-Gaussian marginal distributions using the idea of expectiles. We derive theoretical properties of the proposed processes. Moreover, we use the proposed spatial processes to formulate a spatial regression model for point-referenced data where the spatially correlated errors have skewed marginal distribution. We introduce a Bayesian computational procedure for model fitting and inference for this class of spatial regression models. We compare the performance of the proposed method with the traditional Gaussian process-based spatial regression through simulation studies and by applying it to a dataset on air pollution in California.     

A Multiresolution Tree-Structured Spatial Linear Model

Multiresolution spatial models are able to capture complex dependence correlation in spatial data and are excellent alternatives to the traditional random field models for mapping spatial processes. Because of the multiresolution structures, spatial process prediction can be obtained by direct and fast computation algorithms. However, the existing multiresolution models usually assume a simple constant mean structure, which may not be suitable in practice. In this article, we focus on a multiresolution tree-structured spatial model and extend the model to incorporate a linear regression mean. We explore the properties of the multiresolution tree-structured spatial linear model in depth and estimate the parameters in the linear regression mean and the spatial-dependence structure simultaneously. An expectation-maximization algorithm is adopted to obtain the maximum likelihood estimates of the model parameters and the corresponding information matrix. Given the estimated parameters, a one-pass change-of-resolution Kalman filter algorithm is implemented to obtain the best linear unbiased predictor of the true underlying spatial process. For illustration, the methodology is applied to optimally map crop yield in a Wisconsin field, after accounting for the field conditions by a linear regression.     

基于空间自回归模型的缺失值插补方法

本文研究来自于区域的截面数据中缺失值的插补问题,讨论了当数据中存在空间相关时,空间自回归模型的建立以及利用其对缺失值进行插补的方法,并根据实际数据,通过建立模型给出插补结果。     

Approximate Bayesian Computation and Model Assessment for Repulsive Spatial Point Processes

In many applications involving spatial point patterns, we find evidence of inhibition or repulsion. The most commonly used class of models for such settings are the Gibbs point processes. A recent alternative, at least to the statistical community, is the determinantal point process. Here, we examine model fitting and inference for both of these classes of processes in a Bayesian framework. While usual MCMC model fitting can be available, the algorithms are complex and are not always well behaved. We propose using approximate Bayesian computation (ABC) for such fitting. This approach becomes attractive because, though likelihoods are very challenging to work with for these processes, generation of realizations given parameter values is relatively straightforward. As a result, the ABC fitting approach is well-suited for these models. In addition, such simulation makes them well-suited for posterior predictive inference as well as for model assessment. We provide details for all of the above along with some simulation investigation and an illustrative analysis of a point pattern of tree data exhibiting repulsion. R code and datasets are included in the supplementary material.     

Spatial Regression Models,Response Surfaces,and Process Optimization

Abstract

Spatial regression models are developed as a complementary alternative to second-order polynomial response surfaces in the context of process optimization. These models provide estimates of design variable effects and smooth, data-faithful approximations to the unknown response function over the design space. The predicted response surfaces are driven by the covariance structures of the models. Several structures, isotropic and anisotropic, are considered and connections with thin plate splines are reviewed. Estimation of covariance parameters is achieved via maximum likelihood and residual maximum likelihood. A feature of the spatial regression approach is the visually appealing graphical summaries that are produced. These allow rapid and intuitive identification of process windows on the design space for which the response achieves target performance. Relevant design issues are briefly discussed and spatial designs, such as the packing designs available in Gosset, are suggested as a suitable design complement. The spatial regression models also perform well with no global design, for example with data obtained from series of designs on the same space of design variables. The approach is illustrated with an example involving the optimization of components in a DNA amplification assay. A Monte Carlo comparison of the spatial models with both thin plate splines and second-order polynomial response surfaces for a scenario motivated by the example is also given. This shows superior performance of the spatial models to the second-order polynomials with respect to both prediction over the complete design space and for cross-validation prediction error in the region of the optimum. An anisotropic spatial regression model performs best for a high noise case and both this model and the thin plate spline for a low noise case. Spatial regression is recommended for construction of response surfaces in all process optimization applications.     

Bayesian analysis of conditional autoregressive models

Conditional autoregressive (CAR) models have been extensively used for the analysis of spatial data in diverse areas, such as demography, economy, epidemiology and geography, as models for both latent and observed variables. In the latter case, the most common inferential method has been maximum likelihood, and the Bayesian approach has not been used much. This work proposes default (automatic) Bayesian analyses of CAR models. Two versions of Jeffreys prior, the independence Jeffreys and Jeffreys-rule priors, are derived for the parameters of CAR models and properties of the priors and resulting posterior distributions are obtained. The two priors and their respective posteriors are compared based on simulated data. Also, frequentist properties of inferences based on maximum likelihood are compared with those based on the Jeffreys priors and the uniform prior. Finally, the proposed Bayesian analysis is illustrated by fitting a CAR model to a phosphate dataset from an archaeological region.     

Multivariate spatial regression models

This paper describes the inference procedures required to perform Bayesian inference to some multivariate econometric models. These models have a spatial component built into commonly used multivariate models. In particular, the common component models are addressed and extended to accommodate for spatial dependence. Inference procedures are based on a variety of simulation-based schemes designed to obtain samples from the posterior distribution of model parameters. They are also used to provide a basis to forecast new observations.     

Objective Bayesian analysis for CAR models

Objective priors, especially reference priors, have been studied extensively for spatial data in the last decade. In this paper, we study objective priors for a CAR model. In particular, the properties of the reference prior and the corresponding posterior are studied. Furthermore, we show that the frequentist coverage probabilities of posterior credible intervals depend only on the spatial dependence parameter $\rho $ , and not on the regression coefficient or the error variance. Based on the simulation study for comparing the reference and Jeffreys priors, the performance of two reference priors is similar and better than the Jeffreys priors. One spatial dataset is used for illustration.     

High-Order Composite Likelihood Inference for Max-Stable Distributions and Processes

In multivariate or spatial extremes, inference for max-stable processes observed at a large collection of points is a very challenging problem and current approaches typically rely on less expensive composite likelihoods constructed from small subsets of data. In this work, we explore the limits of modern state-of-the-art computational facilities to perform full likelihood inference and to efficiently evaluate high-order composite likelihoods. With extensive simulations, we assess the loss of information of composite likelihood estimators with respect to a full likelihood approach for some widely used multivariate or spatial extreme models, we discuss how to choose composite likelihood truncation to improve the efficiency, and we also provide recommendations for practitioners. This article has supplementary material online.     

A primer on disease mapping and ecological regression using {\texttt{INLA}}

Spatial and spatio-temporal disease mapping models are widely used for the analysis of registry data and usually formulated in a hierarchical Bayesian framework. Explanatory variables can be included by a so-called ecological regression. It is possible to assume both a linear and a nonparametric association between disease incidence and the explanatory variable. Integrated nested Laplace approximations (INLA) can be used as a tool for Bayesian inference. INLA is a promising alternative to Markov chain Monte Carlo (MCMC) methods which provides very accurate results within short computational time. It is shown in this paper, how parameter estimates for well-known spatial and spatio-temporal models can be obtained by running INLA directly in R{\texttt{R}} using the package INLA{\texttt{INLA}}. Selected R{\texttt{R}} code is shown. An emphasis is given to the inclusion of an explanatory variable. Cases of Coxiellosis among Swiss cows from 2005 to 2008 are used for illustration. The number of stillborn calves is included as time-varying covariate. Additionally, various aspects of INLA such as model choice criteria, computer time, accuracy of the results and usability of the R{\texttt{R}} package are discussed.     

Bayesian Inference for the One-Factor Copula Model

We develop efficient Bayesian inference for the one-factor copula model with two significant contributions over existing methodologies. First, our approach leads to straightforward inference on dependence parameters and the latent factor; only inference on the former is available under frequentist alternatives. Second, we develop a reversible jump Markov chain Monte Carlo algorithm that averages over models constructed from different bivariate copula building blocks. Our approach accommodates any combination of discrete and continuous margins. Through extensive simulations, we compare the computational and Monte Carlo efficiency of alternative proposed sampling schemes. The preferred algorithm provides reliable inference on parameters, the latent factor, and model space. The potential of the methodology is highlighted in an empirical study of 10 binary measures of socio-economic deprivation collected for 11,463 East Timorese households. The importance of conducting inference on the latent factor is motivated by constructing a poverty index using estimates of the factor. Compared to a linear Gaussian factor model, our model average improves out-of-sample fit. The relationships between the poverty index and observed variables uncovered by our approach are diverse and allow for a richer and more precise understanding of the dependence between overall deprivation and individual measures of well-being.     

A higher order Markov model for analyzing covariate dependence

During the recent past, there has been a renewed interest in Markov chain for its attractive properties for analyzing real life data emerging from time series or longitudinal data in various fields. The models were proposed for fitting first or higher order Markov chains. However, there is a serious lack of realistic methods for linking covariate dependence with transition probabilities in order to analyze the factors associated with such transitions especially for higher order Markov chains. L.R. Muenz and L.V. Rubinstein [Markov models for covariate dependence of binary sequences, Biometrics 41 (1985) 91–101] employed logistic regression models to analyze the transition probabilities for a first order Markov model. The methodology is still far from generalization in terms of formulating a model for higher order Markov chains. In this study, it is aimed to provide a comprehensive covariate-dependent Markov model for higher order. The proposed model generalizes the estimation procedure for Markov models for any order. The proposed models and inference procedures are simple and the covariate dependence of the transition probabilities of any order can be examined without making the underlying model complex. An example from rainfall data is illustrated in this paper that shows the utility of the proposed model for analyzing complex real life problems. The application of the proposed method indicates that the higher order covariate dependent Markov models can be conveniently employed in a very useful manner and the results can provide in-depth insights to both the researchers and policymakers to resolve complex problems of underlying factors attributing to different types of transitions, reverse transitions and repeated transitions. The estimation and test procedures can be employed for any order of Markov model without making the theory and interpretation difficult for the common users.     

Gauss–Seidel Estimation of Generalized Linear Mixed Models With Application to Poisson Modeling of Spatially Varying Disease Rates

Generalized linear mixed models (GLMMs) are often fit by computational procedures such as penalized quasi-likelihood (PQL). Special cases of GLMMs are generalized linear models (GLMs), which are often fit using algorithms like iterative weighted least squares (IWLS). High computational costs and memory space constraints make it difficult to apply these iterative procedures to datasets having a very large number of records.

We propose a computationally efficient strategy based on the Gauss–Seidel algorithm that iteratively fits submodels of the GLMM to collapsed versions of the data. The strategy is applied to investigate the relationship between ischemic heart disease, socioeconomic status, and age/gender category in New South Wales, Australia, based on outcome data consisting of approximately 33 million records. For Poisson and binomial regression models, the Gauss–Seidel approach is found to substantially outperform existing methods in terms of maximum analyzable sample size. Remarkably, for both models, the average time per iteration and the total time until convergence of the Gauss–Seidel procedure are less than 0.3% of the corresponding times for the IWLS algorithm. Platform-independent pseudo-code for fitting GLMS, as well as the source code used to generate and analyze the datasets in the simulation studies, are available online as supplemental materials.     

Computation of Spatial Covariance Matrices

In an earlier article, Ghosh derived the density for the distance between two points uniformly and independently distributed in a rectangle. This article extends that work to include the case where the two points lie in two different rectangles in a lattice. This density allows one to find the expected value of certain functions of this distance between rectangles analytically or by one-dimensional numerical integration.

In the case of isotropic spatial models or spatial models with geometric anisotropy terms for agricultural experiments one can use these theoretical results to compute the covariance between the yields in different rectangular plots. As the numerical integration is one-dimensional these results are computed quickly and accurately. The types of covariance functions used come from the Matérn and power families of processes. Analytic results are derived for the de Wijs process, a member of both families and for the power models also.

Software in R is available. Examples of the code are given for fitting spatial models to the Fairfield Smith data. Other methods for the estimation of the covariance matrices are discussed and their pros and cons are outlined.     

Differentiating between good credits and bad credits using neuro-fuzzy systems

To evaluate consumer loan applications, loan officers use many techniques such as judgmental systems, statistical models, or simply intuitive experience. In recent years, fuzzy systems and neural networks have attracted the growing interest of researchers and practitioners. This study compares the performance of artificial neuro-fuzzy inference systems (ANFIS) and multiple discriminant analysis models to screen potential defaulters on consumer loans. Using a modeling sample and a test sample, we find that the neuro-fuzzy system performs better than the multiple discriminant analysis approach to identify bad credit applications. Further, neuro-fuzzy systems have many advantages over traditional computational methods. Neuro-fuzzy system models are flexible, more tolerant of imprecise data, and can model non-linear functions of arbitrary complexity.