Parallel Variational Bayes for Large Datasets With an Application to Generalized Linear Mixed Models

The article develops a hybrid variational Bayes (VB) algorithm that combines the mean-field and stochastic linear regression fixed-form VB methods. The new estimation algorithm can be used to approximate any posterior without relying on conjugate priors. We propose a divide and recombine strategy for the analysis of large datasets, which partitions a large dataset into smaller subsets and then combines the variational distributions that have been learned in parallel on each separate subset using the hybrid VB algorithm. We also describe an efficient model selection strategy using cross-validation, which is straightforward to implement as a by-product of the parallel run. The proposed method is applied to fitting generalized linear mixed models. The computational efficiency of the parallel and hybrid VB algorithm is demonstrated on several simulated and real datasets. Supplementary material for this article is available online.     

Dynamically Updated Spatially Varying Parameterizations of Hierarchical Bayesian Models for Spatial Data

Fitting hierarchical Bayesian models to spatially correlated datasets using Markov chain Monte Carlo (MCMC) techniques is computationally expensive. Complicated covariance structures of the underlying spatial processes, together with high-dimensional parameter space, mean that the number of calculations required grows cubically with the number of spatial locations at each MCMC iteration. This necessitates the need for efficient model parameterizations that hasten the convergence and improve the mixing of the associated algorithms. We consider partially centred parameterizations (PCPs) which lie on a continuum between what are known as the centered (CP) and noncentered parameterizations (NCP). By introducing a weight matrix we remove the conditional posterior correlation between the fixed and the random effects, and hence construct a PCP which achieves immediate convergence for a three-stage model, based on multiple Gaussian processes with known covariance parameters. When the covariance parameters are unknown we dynamically update the parameterization within the sampler. The PCP outperforms both the CP and the NCP and leads to a fully automated algorithm which has been demonstrated in two simulation examples. The effectiveness of the spatially varying PCP is illustrated with a practical dataset of nitrogen dioxide concentration levels. Supplemental materials consisting of appendices, datasets, and computer code to reproduce the results are available online.     

Laplace Variational Approximation for Semiparametric Regression in the Presence of Heteroscedastic Errors

Variational approximations provide fast, deterministic alternatives to Markov chain Monte Carlo for Bayesian inference on the parameters of complex, hierarchical models. Variational approximations are often limited in practicality in the absence of conjugate posterior distributions. Recent work has focused on the application of variational methods to models with only partial conjugacy, such as in semiparametric regression with heteroscedastic errors. Here, both the mean and log variance functions are modeled as smooth functions of covariates. For this problem, we derive a mean field variational approximation with an embedded Laplace approximation to account for the nonconjugate structure. Empirical results with simulated and real data show that our approximate method has significant computational advantages over traditional Markov chain Monte Carlo; in this case, a delayed rejection adaptive Metropolis algorithm. The variational approximation is much faster and eliminates the need for tuning parameter selection, achieves good fits for both the mean and log variance functions, and reasonably reflects the posterior uncertainty. We apply the methods to log-intensity data from a small angle X-ray scattering experiment, in which properly accounting for the smooth heteroscedasticity leads to significant improvements in posterior inference for key physical characteristics of an organic molecule.     

Statistically and Computationally Efficient Estimating Equations for Large Spatial Datasets

For Gaussian process models, likelihood-based methods are often difficult to use with large irregularly spaced spatial datasets, because exact calculations of the likelihood for n observations require O(n³) operations and O(n²) memory. Various approximation methods have been developed to address the computational difficulties. In this article, we propose new, unbiased estimating equations (EE) based on score equation approximations that are both computationally and statistically efficient. We replace the inverse covariance matrix that appears in the score equations by a sparse matrix to approximate the quadratic forms, then set the resulting quadratic forms equal to their expected values to obtain unbiased EE. The sparse matrix is constructed by a sparse inverse Cholesky approach to approximate the inverse covariance matrix. The statistical efficiency of the resulting unbiased EE is evaluated both in theory and by numerical studies. Our methods are applied to nearly 90,000 satellite-based measurements of water vapor levels over a region in the Southeast Pacific Ocean.     

Regularity of solutions to a contact problem

We consider a variational inequality for the Lamé system which models an elastic body in contact with a rigid foundation. We give conditions on the domain and the contact set which allow us to prove regularity of solutions to the variational inequality. In particular, we show that the gradient of the solution is a square integrable function on the boundary.

     

Stochastic Variational Inequality Approaches to the Stochastic Generalized Nash Equilibrium with Shared Constraints

In this paper, we consider the generalized Nash equilibrium with shared constraints in the stochastic environment, and we call it the stochastic generalized Nash equilibrium. The stochastic variational inequalities are employed to solve this kind of problems, and the expected residual minimization model and the conditional value-at-risk formulations defined by the residual function for the stochastic variational inequalities are discussed. We show the risk for different kinds of solutions for the stochastic generalized Nash equilibrium by the conditional value-at-risk formulations. The properties of the stochastic quadratic generalized Nash equilibrium are shown. The smoothing approximations for the expected residual minimization formulation and the conditional value-at-risk formulation are employed. Moreover, we establish the gradient consistency for the measurable smoothing functions and the integrable functions under some suitable conditions, and we also analyze the properties of the formulations. Numerical results for the applications arising from the electricity market model illustrate that the solutions for the stochastic generalized Nash equilibrium given by the ERM model have good properties, such as robustness, low risk and so on.     

Adaptive Bayesian Nonstationary Modeling for Large Spatial Datasets Using Covariance Approximations

Gaussian process models have been widely used in spatial statistics but face tremendous modeling and computational challenges for very large nonstationary spatial datasets. To address these challenges, we develop a Bayesian modeling approach using a nonstationary covariance function constructed based on adaptively selected partitions. The partitioned nonstationary class allows one to knit together local covariance parameters into a valid global nonstationary covariance for prediction, where the local covariance parameters are allowed to be estimated within each partition to reduce computational cost. To further facilitate the computations in local covariance estimation and global prediction, we use the full-scale covariance approximation (FSA) approach for the Bayesian inference of our model. One of our contributions is to model the partitions stochastically by embedding a modified treed partitioning process into the hierarchical models that leads to automated partitioning and substantial computational benefits. We illustrate the utility of our method with simulation studies and the global Total Ozone Matrix Spectrometer (TOMS) data. Supplementary materials for this article are available online.     

Covariance tapering for prediction of large spatial data sets in transformed random fields

The best linear unbiased predictor (BLUP) is called a kriging predictor and has been widely used to interpolate a spatially correlated random process in scientific areas such as geostatistics. However, if an underlying random field is not Gaussian, the optimality of the BLUP in the mean squared error (MSE) sense is unclear because it is not always identical with the conditional expectation. Moreover, in many cases, data sets in spatial problems are often so large that a kriging predictor is impractically time-consuming. To reduce the computational complexity, covariance tapering has been developed for large spatial data sets. In this paper, we consider covariance tapering in a class of transformed Gaussian models for random fields and show that the BLUP using covariance tapering, the BLUP and the optimal predictor are asymptotically equivalent in the MSE sense if the underlying Gaussian random field has the Matérn covariance function.     

Error rates in classification of multivariate Gaussian random field observation

We consider the problem of supervised classifying the multivariate Gaussian random field (GRF) single observation into one of two populations in case of given training sample. The populations are specified by different regression mean models and by common factorized covariance function. For completely specified populations, we derive a formula for Bayes error rate. In the case of unknown regression parameters and feature covariance matrix, the plug-in Bayes discriminant function based on ML estimators of parameters is used for classification. We derive the actual error rate and the asymptotic expansion of the expected error rate associated with plug-in Bayes discriminant function. These results are multivariate generalizations of previous ones. Numerical analysis of the derived formulas is implemented for the bivariate GRF observations at locations belonging to the two-dimensional lattice with unit spacing.     

Variational Bayes approximations for clustering via mixtures of normal inverse Gaussian distributions

Parameter estimation for model-based clustering using a finite mixture of normal inverse Gaussian (NIG) distributions is achieved through variational Bayes approximations. Univariate NIG mixtures and multivariate NIG mixtures are considered. The use of variational Bayes approximations here is a substantial departure from the traditional EM approach and alleviates some of the associated computational complexities and uncertainties. Our variational algorithm is applied to simulated and real data. The paper concludes with discussion and suggestions for future work.     

Image denoising using the Gaussian curvature of the image surface

A number of high‐order variational models for image denoising have been proposed within the last few years. The main motivation behind these models is to fix problems such as the staircase effect and the loss of image contrast that the classical Rudin–Osher–Fatemi model [Leonid I. Rudin, Stanley Osher and Emad Fatemi, Nonlinear total variation based noise removal algorithms, Physica D 60 (1992), pp. 259–268] and others also based on the gradient of the image do have. In this work, we propose a new variational model for image denoising based on the Gaussian curvature of the image surface of a given image. We analytically study the proposed model to show why it preserves image contrast, recovers sharp edges, does not transform piecewise smooth functions into piecewise constant functions and is also able to preserve corners. In addition, we also provide two fast solvers for its numerical realization. Numerical experiments are shown to illustrate the good performance of the algorithms and test results. © 2015 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq 32: 1066–1089, 2016     

Quantile estimations via modified Cholesky decomposition for longitudinal single-index models

Quantile regression is a powerful complement to the usual mean regression and becomes increasingly popular due to its desirable properties. In longitudinal studies, it is necessary to consider the intra-subject correlation among repeated measures over time to improve the estimation efficiency. In this paper, we focus on longitudinal single-index models. Firstly, we apply the modified Cholesky decomposition to parameterize the intra-subject covariance matrix and develop a regression approach to estimate the parameters of the covariance matrix. Secondly, we propose efficient quantile estimating equations for the index coefficients and the link function based on the estimated covariance matrix. Since the proposed estimating equations include a discrete indicator function, we propose smoothed estimating equations for fast and accurate computation of the index coefficients, as well as their asymptotic covariances. Thirdly, we establish the asymptotic properties of the proposed estimators. Finally, simulation studies and a real data analysis have illustrated the efficiency of the proposed approach.

     

Example of a Gaussian Self-Similar Field With Stationary Rectangular Increments That Is Not a Fractional Brownian Sheet

We consider anisotropic self-similar random fields, in particular, the fractional Brownian sheet (fBs). This Gaussian field is an extension of fractional Brownian motion. It is well known that the fractional Brownian motion is a unique Gaussian self-similar process with stationary increments. The main result of this article is an example of a Gaussian self-similar field with stationary rectangular increments that is not an fBs. So we proved that the structure of self-similar Gaussian fields can be substantially more involved then the structure of self-similar Gaussian processes. In order to establish the main result, we prove some properties of covariance function for self-similar fields with rectangular increments. Also, using Lamperti transformation, we obtain properties of covariance function for the corresponding stationary fields.     

The cumulative quantile regression function with censored and truncated covariate

In this paper, we consider the empirical estimator of the cumulative quantile regression (CQR) functionwhen the covariate is subjected to random truncation and censorship. Strong Gaussian approximations for the associated CQR process are established under appropriate assumptions. A functional law of the iterated logarithm for the CQR process is also derived. These results provide a foundation for the asymptotic theory of functional statistics based on these processes.     

期权定价问题的数值方法

本文研究以股票为标的资产的美式看跌期权定价问题的数值方法,即有限元方法。通过将所考虑的问题转化为等价的变分不等式,并利用积分恒等式与超逼近分析技术,得到了半离散有限元方法的最优L~2-模与L~∞-模的误差估计。     

Dynamical systems and variational inequalities

The variational inequality problem has been utilized to formulate and study a plethora of competitive equilibrium problems in different disciplines, ranging from oligopolistic market equilibrium problems to traffic network equilibrium problems. In this paper we consider for a given variational inequality a naturally related ordinary differential equation. The ordinary differential equations that arise are nonstandard because of discontinuities that appear in the dynamics. These discontinuities are due to the constraints associated with the feasible region of the variational inequality problem. The goals of the paper are two-fold. The first goal is to demonstrate that although non-standard, many of the important quantitative and qualitative properties of ordinary differential equations that hold under the standard conditions, such as Lipschitz continuity type conditions, apply here as well. This is important from the point of view of modeling, since it suggests (at least under some appropriate conditions) that these ordinary differential equations may serve as dynamical models. The second goal is to prove convergence for a class of numerical schemes designed to approximate solutions to a given variational inequality. This is done by exploiting the equivalence between the stationary points of the associated ordinary differential equation and the solutions of the variational inequality problem. It can be expected that the techniques described in this paper will be useful for more elaborate dynamical models, such as stochastic models, and that the connection between such dynamical models and the solutions to the variational inequalities will provide a deeper understanding of equilibrium problems.     

A moving average Cholesky factor model in joint mean-covariance modeling for longitudinal data

Modeling the mean and covariance simultaneously is a common strategy to efciently estimate the mean parameters when applying generalized estimating equation techniques to longitudinal data.In this article,using generalized estimation equation techniques,we propose a new kind of regression models for parameterizing covariance structures.Using a novel Cholesky factor,the entries in this decomposition have moving average and log innovation interpretation and are modeled as linear functions of covariates.The resulting estimators for the regression coefcients in both the mean and the covariance are shown to be consistent and asymptotically normally distributed.Simulation studies and a real data analysis show that the proposed approach yields highly efcient estimators for the parameters in the mean,and provides parsimonious estimation for the covariance structure.     

A comparative study of Gaussian geostatistical models and Gaussian Markov random field models

Gaussian geostatistical models (GGMs) and Gaussian Markov random fields (GMRFs) are two distinct approaches commonly used in spatial models for modeling point-referenced and areal data, respectively. In this paper, the relations between GGMs and GMRFs are explored based on approximations of GMRFs by GGMs, and approximations of GGMs by GMRFs. Two new metrics of approximation are proposed : (i) the Kullback-Leibler discrepancy of spectral densities and (ii) the chi-squared distance between spectral densities. The distances between the spectral density functions of GGMs and GMRFs measured by these metrics are minimized to obtain the approximations of GGMs and GMRFs. The proposed methodologies are validated through several empirical studies. We compare the performance of our approach to other methods based on covariance functions, in terms of the average mean squared prediction error and also the computational time. A spatial analysis of a dataset on PM_2.5 collected in California is presented to illustrate the proposed method.     

System of nonsmooth variational inequalities with applications

In this paper, we consider a system of vector variational inequalities and a system of nonsmooth variational inequalities defined by means of Clarke directional derivative. We also consider the Nash equilibrium problem with vector pay-offs and its scalarized form. We present some relations among these systems and problems. The existence results for a solution of system of nonsmooth variational inequalities are given. As a consequence, we derive an existence result for a solution of Nash equilibrium problem with vector pay-offs.     

Lexicographic variational inequalities with applications

In this article, we consider equivalence properties between various kinds of lexicographic variational inequalities. By employing various concepts of monotonicity, we show that the usual sequential variational inequality is equivalent to the direct lexicographic variational inequality or to the dual lexicographic variational inequality. We establish several existence results for lexicographic variational inequalities. Also, we introduce the lexicographic complementarity problem and establish its equivalence with the lexicographic variational inequality. We illustrate our approach by several examples of applications to vector transportation and vector spatial equilibrium problems.