Sensitivity analysis for inference with partially identifiable covariance matrices

In some multivariate problems with missing data, pairs of variables exist that are never observed together. For example, some modern biological tools can produce data of this form. As a result of this structure, the covariance matrix is only partially identifiable, and point estimation requires that identifying assumptions be made. These assumptions can introduce an unknown and potentially large bias into the inference. This paper presents a method based on semidefinite programming for automatically quantifying this potential bias by computing the range of possible equal-likelihood inferred values for convex functions of the covariance matrix. We focus on the bias of missing value imputation via conditional expectation and show that our method can give an accurate assessment of the true error in cases where estimates based on sampling uncertainty alone are overly optimistic.     

The population-sample decomposition approach to multivariate estimation methods

In this paper it is argued that all multivariate estimation methods, such as OLS regression, simultaneous linear equations systems and, more widely, what are known as LISREL methods, have merit as geometric approximation methods, even if the observations are not drawn from a multivariate normal parent distribution and consequently cannot be viewed as ML estimators. It is shown that for large samples the asymptotical distribution of any estimator, being a totally differentiable covariance function, may be assessed by the δ method. Finally, we stress that the design of the sample and a priori knowledge about the parent distribution may be incorporated to obtain more specific results. It turns out that some fairly traditional assumptions, such as assuming some variables to be non-random, fixed over repeated samples, or the existence of a parent normal distribution, may have dramatic effects on the assessment of standard deviations and confidence bounds, if such assumptions are not realistic. The method elaborated by us does not make use of such assumptions.     

Multivariate Adaptive Splines for Analysis of Longitudinal Data

Abstract

An essential feature of longitudinal data is the existence of autocorrelation among the observations from the same unit or subject. Two-stage random-effects linear models are commonly used to analyze longitudinal data. These models are not flexible enough, however, for exploring the underlying data structures and, especially, for describing time trends. Semi-parametric models have been proposed recently to accommodate general time trends. But these semi-parametric models do not provide a convenient way to explore interactions among time and other covariates although such interactions exist in many applications. Moreover, semi-parametric models require specifying the design matrix of the covariates (time excluded). We propose nonparametric models to resolve these issues. To fit nonparametric models, we use the novel technique of the multivariate adaptive regression splines for the estimation of mean curve and then apply an EM-like iterative procedure for covariance estimation. After giving a general algorithm of model building, we show how to design a fast algorithm. We use both simulated and published data to illustrate the use of our proposed method.     

Estimation of covariance matrices in fixed and mixed effects linear models

The estimation of the covariance matrix or the multivariate components of variance is considered in the multivariate linear regression models with effects being fixed or random. In this paper, we propose a new method to show that usual unbiased estimators are improved on by the truncated estimators. The method is based on the Stein–Haff identity, namely the integration by parts in the Wishart distribution, and it allows us to handle the general types of scale-equivariant estimators as well as the general fixed or mixed effects linear models.     

Efficient parameter estimation via modified Cholesky decomposition for quantile regression with longitudinal data

It is well known that specifying a covariance matrix is difficult in the quantile regression with longitudinal data. This paper develops a two step estimation procedure to improve estimation efficiency based on the modified Cholesky decomposition. Specifically, in the first step, we obtain the initial estimators of regression coefficients by ignoring the possible correlations between repeated measures. Then, we apply the modified Cholesky decomposition to construct the covariance models and obtain the estimator of within-subject covariance matrix. In the second step, we construct unbiased estimating functions to obtain more efficient estimators of regression coefficients. However, the proposed estimating functions are discrete and non-convex. We utilize the induced smoothing method to achieve the fast and accurate estimates of parameters and their asymptotic covariance. Under some regularity conditions, we establish the asymptotically normal distributions for the resulting estimators. Simulation studies and the longitudinal progesterone data analysis show that the proposed approach yields highly efficient estimators.     

An alternative method for computing mean and covariance matrix of some multivariate distributions

Computing the mean and covariance matrix of some multivariate distributions, in particular, multivariate normal distribution and Wishart distribution are considered in this article. It involves a matrix transformation of the normal random vector into a random vector whose components are independent normal random variables, and then integrating univariate integrals for computing the mean and covariance matrix of a multivariate normal distribution. Moment generating function technique is used for computing the mean and covariances between the elements of a Wishart matrix. In this article, an alternative method that uses matrix differentiation and differentiation of the determinant of a matrix is presented. This method does not involve any integration.     

误差服从多元t分布的一类线性模型下参数估计的若干注记

研究一类线性模型下参数估计的若干问题．这类模型包含了多个因变量线性模型、增长曲线模型、扩充的增长曲线模型、似乎不相关回归方程组、方差分量模型等常用模型．在这类线性模型下,证明了当误差服从多元t分布时与误差服从多元正态分布时,具有相同的完全统计量和无偏估计,且在后一种情况下的充分统计量必为前一种情况下的充分统计量．对于带有多种协方差结构的前述几种模型,把在误差服从多元正态分布下,相应的协方差阵及有关参数的一致最小风险无偏(UMRU)估计存在性的结论推广到了相应的误差服从多元t分布情形．此外,对于误差服从多元t分布的这类统一的线性模型,给出了回归系数的线性可估函数的无偏估计的协方差阵的C-R下界．     

Nonparametric variable selection and its application to additive models

Variable selection for multivariate nonparametric regression models usually involves parameterized approximation for nonparametric functions in the objective function. However, this parameterized approximation often increases the number of parameters significantly, leading to the “curse of dimensionality” and inaccurate estimation. In this paper, we propose a novel and easily implemented approach to do variable selection in nonparametric models without parameterized approximation, enabling selection consistency to be achieved. The proposed method is applied to do variable selection for additive models. A two-stage procedure with selection and adaptive estimation is proposed, and the properties of this method are investigated. This two-stage algorithm is adaptive to the smoothness of the underlying components, and the estimation consistency can reach a parametric rate if the underlying model is really parametric. Simulation studies are conducted to examine the performance of the proposed method. Furthermore, a real data example is analyzed for illustration.

     

Robust estimation in joint mean–covariance regression model for longitudinal data

In this paper, we develop robust estimation for the mean and covariance jointly for the regression model of longitudinal data within the framework of generalized estimating equations (GEE). The proposed approach integrates the robust method and joint mean–covariance regression modeling. Robust generalized estimating equations using bounded scores and leverage-based weights are employed for the mean and covariance to achieve robustness against outliers. The resulting estimators are shown to be consistent and asymptotically normally distributed. Simulation studies are conducted to investigate the effectiveness of the proposed method. As expected, the robust method outperforms its non-robust version under contaminations. Finally, we illustrate by analyzing a hormone data set. By downweighing the potential outliers, the proposed method not only shifts the estimation in the mean model, but also shrinks the range of the innovation variance, leading to a more reliable estimation in the covariance matrix.     

含高维相依自变量的中心k阶条件矩子空间的估计

在回归分析中往往对条件均值,条件方差及高阶条件矩特别感兴趣.本文我们将关注中心k阶条件矩子空间在高维相依自变量情形的估计问题.为此,我们首先引入中心k阶条件矩子空间的概念,并研究该子空间的基本性质.针对高维相依自变量的复杂数据,为了避免预测变量协方差阵的逆矩阵的计算,本文提出用偏最小二乘方法来估计中心k阶条件矩子空间....     

Multivariate nonlinear least squares: robustness and efficiency of standard versus Beauchamp and Cornell methodologies

Simultaneous estimation in nonlinear multivariate regression contexts is a complex problem in inference. In this paper, we compare the methodology suggested in the literature for an unknown covariance matrix among response components, the methodology by Beauchamp and Cornell (B&C), with the standard nonlinear least squares approach (NLS). In the first part of the paper, we contrast B&C and the standard NLS, pointing out, from the theoretical point of view, how a model specification error could affect the estimation. A comprehensive simulation study is also performed to evaluate the effectiveness of B&C versus standard NLS under both correct and misspecified models. Several alternative models are considered to highlight the consequences of different types of specification error. An application to a real dataset within the context of quantitative marketing is presented.     

Adaptive Bayesian Nonstationary Modeling for Large Spatial Datasets Using Covariance Approximations

Gaussian process models have been widely used in spatial statistics but face tremendous modeling and computational challenges for very large nonstationary spatial datasets. To address these challenges, we develop a Bayesian modeling approach using a nonstationary covariance function constructed based on adaptively selected partitions. The partitioned nonstationary class allows one to knit together local covariance parameters into a valid global nonstationary covariance for prediction, where the local covariance parameters are allowed to be estimated within each partition to reduce computational cost. To further facilitate the computations in local covariance estimation and global prediction, we use the full-scale covariance approximation (FSA) approach for the Bayesian inference of our model. One of our contributions is to model the partitions stochastically by embedding a modified treed partitioning process into the hierarchical models that leads to automated partitioning and substantial computational benefits. We illustrate the utility of our method with simulation studies and the global Total Ozone Matrix Spectrometer (TOMS) data. Supplementary materials for this article are available online.     

Bayesian shrinkage prediction for the regression problem

We consider Bayesian shrinkage predictions for the Normal regression problem under the frequentist Kullback-Leibler risk function.Firstly, we consider the multivariate Normal model with an unknown mean and a known covariance. While the unknown mean is fixed, the covariance of future samples can be different from that of training samples. We show that the Bayesian predictive distribution based on the uniform prior is dominated by that based on a class of priors if the prior distributions for the covariance and future covariance matrices are rotation invariant.Then, we consider a class of priors for the mean parameters depending on the future covariance matrix. With such a prior, we can construct a Bayesian predictive distribution dominating that based on the uniform prior.Lastly, applying this result to the prediction of response variables in the Normal linear regression model, we show that there exists a Bayesian predictive distribution dominating that based on the uniform prior. Minimaxity of these Bayesian predictions follows from these results.     

Limiting optimal adaptive filtering with unknown disturbance covariance

The accuracy of estimating the variance of the Kalman-Bucy filter depends essentially on disturbance covariance matrices and measurement noise. The main difficulty in filter design is the lack of necessary statistical information about the useful signal and the disturbance. Filters whose parameters are tuned during active estimation are classified with adaptive filters. The problem of adaptive filtering under parametric uncertainty conditions is studied. A method for designing limiting optimal Kalman-Bucy filters in the case of unknown disturbance covariance is presented. An adaptive algorithm for estimating disturbance covariance matrices based on stochastic approximation is described. Convergence conditions for this algorithm are investigated. The operation of a limiting adaptive filter is exemplified.     

�������ݾ�ֵ��Э����������Ч�Ƚ�����

In this article, we develop efficient robust method for estimation of mean and covariance simultaneously for longitudinal data in regression model. Based on Cholesky decomposition for the covariance matrix and rewriting the regression model, we propose a weighted least square estimator, in which the weights are estimated under generalized empirical likelihood framework. The proposed estimator obtains high efficiency from the close connection to empirical likelihood method, and achieves robustness by bounding the weighted sum of squared residuals. Simulation study shows that, compared to existing robust estimation methods for longitudinal data, the proposed estimator has relatively high efficiency and comparable robustness. In the end, the proposed method is used to analyse a real data set.     

Fuzzy projection pursuit density estimation by eigenvalue method

This paper focuses on the use of kernel method and projection pursuit regression for non-parametric probability density estimation. Direct application of the kernel method is not able to pick up characteristic features of multidimensional density function. We propose a fuzzy projection pursuit density estimation based on the membership function and the eigenvector of the covariance matrix. Marginal densities along the subspace spanned by the projection vector are estimated. The proposed projection pursuit is one of the methods which are able to bypass the ‘curse of dimensionality’ in multidimensional density estimation. An application to experimental design for machining accuracy of end milling with the tool in small diameter is presented to demonstrate its usefulness.     

Using sliced mean variance–covariance inverse regression for classification and dimension reduction

The sliced mean variance–covariance inverse regression (SMVCIR) algorithm takes grouped multivariate data as input and transforms it to a new coordinate system where the group mean, variance, and covariance differences are more apparent. Other popular algorithms used for performing graphical group discrimination are sliced average variance estimation (SAVE, targetting the same differences but using a different arrangement for variances) and sliced inverse regression (SIR, which targets mean differences). We provide an improved SMVCIR algorithm and create a dimensionality test for the SMVCIR coordinate system. Simulations corroborating our theoretical results and comparing SMVCIR with the other methods are presented. We also provide examples demonstrating the use of SMVCIR and the other methods, in visualization and group discrimination by k-nearest neighbors. The advantages and differences of SMVCIR from SAVE and SIR are shown clearly in these examples and simulation.     

多元时间序列模型协方差矩阵的递归算法

本文推导了多元时序模型的协方差矩阵与模型参数的关系式，并给出了计算多维时序过程自协方差矩阵的递归算法     

Curve forecasting by functional autoregression

This paper deals with the prediction of curve-valued autoregression processes. It develops a novel technique, predictive factor decomposition, for the estimation of the autoregression operator. The technique is based on finding a reduced-rank approximation to the autoregression operator that minimizes the expected squared norm of the prediction error.Implementing this idea, we relate the operator approximation problem to the singular value decomposition of a combination of cross-covariance and covariance operators. We develop an estimation method based on regularization of the empirical counterpart of this singular value decomposition, prove its consistency and evaluate convergence rates.The method is illustrated by an example of the term structure of the Eurodollar futures rates. In the sample corresponding to the period of normal growth, the predictive factor technique outperforms the principal components method and performs on a par with custom-designed prediction methods.     

Asymptotics of estimators for nonparametric multivariate regression models with long memory

In this paper, a nonparametric multivariate regression model with long memory covariates and long memory errors is considered. We approximate the nonparametric multivariate regression function by the weighted additive one-dimensional functions. The local linear smoothing and least squares method are proposed for the one-dimensional regression estimation and the weight parameters estimation, respectively. The asymptotic behaviors of the proposed estimators are investigated.