Simulation of the Matrix Bingham–von Mises–Fisher Distribution,With Applications to Multivariate and Relational Data

Orthonormal matrices play an important role in reduced-rank matrix approximations and the analysis of matrix-valued data. A matrix Bingham–von Mises–Fisher distribution is a probability distribution on the set of orthonormal matrices that includes linear and quadratic terms in the log-density, and arises as a posterior distribution in latent factor models for multivariate and relational data. This article describes rejection and Gibbs sampling algorithms for sampling from this family of distributions, and illustrates their use in the analysis of a protein–protein interaction network. Supplemental materials, including code and data to generate all of the numerical results in this article, are available online.     

Bayesian analysis of non-linear structural equation models with non-ignorable missing outcomes from reproductive dispersion models

Non-linear structural equation models are widely used to analyze the relationships among outcomes and latent variables in modern educational, medical, social and psychological studies. However, the existing theories and methods for analyzing non-linear structural equation models focus on the assumptions of outcomes from an exponential family, and hence can’t be used to analyze non-exponential family outcomes. In this paper, a Bayesian method is developed to analyze non-linear structural equation models in which the manifest variables are from a reproductive dispersion model (RDM) and/or may be missing with non-ignorable missingness mechanism. The non-ignorable missingness mechanism is specified by a logistic regression model. A hybrid algorithm combining the Gibbs sampler and the Metropolis–Hastings algorithm is used to obtain the joint Bayesian estimates of structural parameters, latent variables and parameters in the logistic regression model, and a procedure calculating the Bayes factor for model comparison is given via path sampling. A goodness-of-fit statistic is proposed to assess the plausibility of the posited model. A simulation study and a real example are presented to illustrate the newly developed Bayesian methodologies.     

Hierarchical mixture models for zero-inflated correlated count data

Count data with excess zeros are often encountered in many medical, biomedical and public health applications. In this paper, an extension of zero-inflated Poisson mixed regression models is presented for dealing with multilevel data set, referred as hierarchical mixture zero-inflated Poisson mixed regression models. A stochastic EM algorithm is developed for obtaining the ML estimates of interested parameters and a model comparison is also considered for comparing models with different latent classes through BIC criterion. An application to the analysis of count data from a Shanghai Adolescence Fitness Survey and a simulation study illustrate the usefulness and effectiveness of our methodologies.     

On the exploration of linear latent effect for multivariate modeling

A data analysis method is proposed to derive a latent structure matrix from a sample covariance matrix. The matrix can be used to explore the linear latent effect between two sets of observed variables. Procedures with which to estimate a set of dependent variables from a set of explanatory variables by using latent structure matrix are also proposed. The proposed method can assist the researchers in improving the effectiveness of the SEM models by exploring the latent structure between two sets of variables. In addition, a structure residual matrix can also be derived as a by-product of the proposed method, with which researchers can conduct experimental procedures for variables combinations and selections to build various models for hypotheses testing. These capabilities of data analysis method can improve the effectiveness of traditional SEM methods in data property characterization and models hypotheses testing. Case studies are provided to demonstrate the procedure of deriving latent structure matrix step by step, and the latent structure estimation results are quite close to the results of PLS regression. A structure coefficient index is suggested to explore the relationships among various combinations of variables and their effects on the variance of the latent structure.     

带有不可忽略缺失数据的半参数再生散度模型的贝叶斯分析

半参数再生散度模型是再生散度模型和半参数回归模型的推广,包括了半参数广义线性模型和广义部分线性模型等特殊类型.讨论的是该模型在响应变量和协变量均存在非随机缺失数据情形下参数的Bayes估计和基于Bayes因子的模型选择问题.在分析中,采用了惩罚样条来估计模型中的非参数成分,并建立了Bayes层次模型;为了解决Gibbs抽样过程中因参数高度相关带来的混合性差以及因维数增加导致出现不稳定性的问题,引入了潜变量做为添加数据并应用了压缩Gibbs抽样方法,改进了收敛性;同时,为了避免计算多重积分,利用了M-H算法估计边缘密度函数后计算Bayes因子,为模型的选择比较提供了一种准则.最后,通过模拟和实例验证了所给方法的有效性.     

Reduced-rank regression for the multivariate linear model

The problem of estimating the regression coefficient matrix having known (reduced) rank for the multivariate linear model when both sets of variates are jointly stochastic is discussed. We show that this problem is related to the problem of deciding how many principal components or pairs of canonical variates to use in any practical situation. Under the assumption of joint normality of the two sets of variates, we give the asymptotic (large-sample) distributions of the various estimated reduced-rank regression coefficient matrices that are of interest. Approximate confidence bounds on the elements of these matrices are then suggested using either the appropriate asymptotic expressions or the jackknife technique.     

Short-Time Linear Response with Reduced-Rank Tangent Map

The recently developed short-time linear response algorithm, which predicts the response of a nonlinear chaotic forced-dissipative system to small external perturbation, yields high precision of the response prediction. However, the computation of the short-time linear response formula with the full rank tangent map can be expensive. Here, a numerical method to potentially overcome the increasing numerical complexity for large scale models with many variables by using the reduced-rank tangent map in the computation is proposed. The conditions for which the short-time linear response approximation with the reduced-rank tangent map is valid are established, and two practical situations are examined, where the response to small external perturbations is predicted for nonlinear chaotic forced-dissipative systems with different dynamical properties.     

A Local Parameterization of Orthogonal and Semi-Orthogonal Matrices with Applications

This article describes a local parameterization of orthogonal and semi-orthogonal matrices. The parameterization leads to a unified approach for obtaining the asymptotic joint distributions of estimators of singular-values and -vectors, and of eigen-values and -vectors. The singular- or eigen-values can have arbitrary multiplicities. The approach is illustrated on principal components analyzes, canonical correlation analysis, inter-battery factory analysis, and reduced-rank regression.     

High-dimensional generation of Bernoulli random vectors

The objective of this paper is to explore different modeling strategies to generate high-dimensional Bernoulli vectors. We discuss the multivariate Bernoulli (MB) distribution, probe its properties and examine three models for generating random vectors. A latent multivariate normal model whose bivariate distributions are approximated with Plackett distributions with univariate normal distributions is presented. A conditional mean model is examined where the conditional probability of success depends on previous history of successes. A mixture of beta distributions is also presented that expresses the probability of the MB vector as a product of correlated binary random variables. Each method has a domain of effectiveness. The latent model offers unpatterned correlation structures while the conditional mean and the mixture model provide computational feasibility for high-dimensional generation of MB vectors.     

Bayesian lasso binary quantile regression

In this paper, a Bayesian hierarchical model for variable selection and estimation in the context of binary quantile regression is proposed. Existing approaches to variable selection in a binary classification context are sensitive to outliers, heteroskedasticity or other anomalies of the latent response. The method proposed in this study overcomes these problems in an attractive and straightforward way. A Laplace likelihood and Laplace priors for the regression parameters are proposed and estimated with Bayesian Markov Chain Monte Carlo. The resulting model is equivalent to the frequentist lasso procedure. A conceptional result is that by doing so, the binary regression model is moved from a Gaussian to a full Laplacian framework without sacrificing much computational efficiency. In addition, an efficient Gibbs sampler to estimate the model parameters is proposed that is superior to the Metropolis algorithm that is used in previous studies on Bayesian binary quantile regression. Both the simulation studies and the real data analysis indicate that the proposed method performs well in comparison to the other methods. Moreover, as the base model is binary quantile regression, a much more detailed insight in the effects of the covariates is provided by the approach. An implementation of the lasso procedure for binary quantile regression models is available in the R-package bayesQR.     

Least-squares Spline Regression with Block-diagonal Variance Matrices

A numerical method of solution is presented for the least squaresfitting of experimental data by spline functions in the casewhere the data errors are correlated and for which the variancematrix is specified. The method is general in that it permits(a) splines of any order, (b) the knots of the spline to bearbitrary in number and position, and (c) variance matricesthat are block diagonal in form. Since limiting forms of (c)are diagonal and full variance matrices, the method can handle,as special cases, both conventional spline regression problemsand spline regression problems with general, unstructured variancematrices. An application to gamma spectrometry, in which theblocks of the variance matrix have special structure, is fullytreated.     

Trajectory planning based on non-convex global optimization for serial manipulators

To perform specific tasks in dynamic environments, robots are required to rapidly update trajectories according to changing factors. A continuous trajectory planning methodology for serial manipulators based on non-convex global optimization is presented in this paper. First, a kinematic trajectory planning model based on non-convex optimization is constructed to balance motion rapidity and safety. Then, a model transformation method for the non-convex optimization model is presented. In this way, the accurate global solution can be obtained with an iterative solver starting from arbitrary initializations, which can greatly improve the computational accuracy and efficiency. Furthermore, an efficient initialization method for the iterative solver based on multivariable-multiple regression is presented, which further speeds up the solution process. The results show that trajectory planning efficiency is significantly enhanced by model transformation and initialization improvement for the iterative solver. Consequently, real-time continuous trajectory planning for serial manipulators with many degrees of freedom can be achieved, which lays a basis for performing dynamic tasks in complex environments.     

Multiblock latent root regression. Application to epidemiological data

Several papers have already stressed the interest of latent root regression and its similarities to partial least squares regression. A new formulation of this method which makes it even simpler than the original method to set up a prediction model is discussed. Furthermore, it is shown how this method can be extended not only to the case where it is desired to predict several response variables from a set of predictors but also to the multiblock setting where the aim is to predict one or several data sets from several other data sets. The interest of the method is illustrated on the basis of a data set pertaining to epidemiology.     

经济预测中的正交回归分析

本文介绍了一种新的线性模型参数回归分析方法即正交回归，并以建立经济模型为例，对正交回归和经典回归的结果进行了比较。     

A fuzzy theoretic approach to simulation metamodeling

In this paper, we explore the potential application of fuzzy linear regression in developing simulation metamodels. It should be noted that the basic construct for simulation metamodels involves uncertainties and ambiguities that may be better addressed through fuzzy linear regression application. The solution techniques employed by fuzzy linear regression are very familiar, and the generation of fuzzy outputs may offer a wide range of solution space to the decision maker, thereby reducing the risk of making an incorrect economic decision. A numerical example is presented to show how a possibility distribution is used to capture the vagueness in a dependent variable for a regression metamodel.     

A Computationally Efficient Projection-Based Approach for Spatial Generalized Linear Mixed Models

Inference for spatial generalized linear mixed models (SGLMMs) for high-dimensional non-Gaussian spatial data is computationally intensive. The computational challenge is due to the high-dimensional random effects and because Markov chain Monte Carlo (MCMC) algorithms for these models tend to be slow mixing. Moreover, spatial confounding inflates the variance of fixed effect (regression coefficient) estimates. Our approach addresses both the computational and confounding issues by replacing the high-dimensional spatial random effects with a reduced-dimensional representation based on random projections. Standard MCMC algorithms mix well and the reduced-dimensional setting speeds up computations per iteration. We show, via simulated examples, that Bayesian inference for this reduced-dimensional approach works well both in terms of inference as well as prediction; our methods also compare favorably to existing “reduced-rank” approaches. We also apply our methods to two real world data examples, one on bird count data and the other classifying rock types. Supplementary material for this article is available online.     

Mixtures of factor analyzers: an extension with covariates

This paper examines the analysis of an extended finite mixture of factor analyzers (MFA) where both the continuous latent variable (common factor) and the categorical latent variable (component label) are assumed to be influenced by the effects of fixed observed covariates. A polytomous logistic regression model is used to link the categorical latent variable to its corresponding covariate, while a traditional linear model with normal noise is used to model the effect of the covariate on the continuous latent variable. The proposed model turns out be in various ways an extension of many existing related models, and as such offers the potential to address some of the issues not fully handled by those previous models. A detailed derivation of an EM algorithm is proposed for parameter estimation, and latent variable estimates are obtained as by-products of the overall estimation procedure.     

Book Review

A method of estimation of the coefficients of a linear regression model is described as invariant if the basic regression results obtained by the method are unaltered by a location/scale transformation of the data matrix. A necessary and sufficient condition for a method to be invariant is presented. Finally specific methods of estimation are examined for invariance.     

Numerical solution of nonlinear Volterra-Fredholm-Hammerstein integral equations via collocation method based on radial basis functions

A numerical technique based on the spectral method is presented for the solution of nonlinear Volterra-Fredholm-Hammerstein integral equations. This method is a combination of collocation method and radial basis functions (RBFs) with the differentiation process (DRBF), using zeros of the shifted Legendre polynomial as the collocation points. Different applications of RBFs are used for this purpose. The integral involved in the formulation of the problems are approximated based on Legendre-Gauss-Lobatto integration rule. The results of numerical experiments are compared with the analytical solution in illustrative examples to confirm the accuracy and efficiency of the presented scheme.     

Dual approaches to the minimization of strongly convex functionals with a simple structure under affine constraints

A strongly convex function of simple structure (for example, separable) is minimized under affine constraints. A dual problem is constructed and solved by applying a fast gradient method. The necessary properties of this method are established relying on which, under rather general conditions, the solution of the primal problem can be recovered with the same accuracy as the dual solution from the sequence generated by this method in the dual space of the problem. Although this approach seems natural, some previously unpublished rather subtle results necessary for its rigorous and complete theoretical substantiation in the required generality are presented.