Restricted estimation in multivariate measurement error regression model

We study a multivariate ultrastructural measurement error (MUME) model with more than one response variable. This model is a synthesis of multivariate functional and structural models. Three consistent estimators of regression coefficients, satisfying the exact linear restrictions have been proposed. Their asymptotic distributions are derived under the assumption of a non-normal measurement error and random error components. A simulation study is carried out to investigate the small sample properties of the estimators. The effect of departure from normality of the measurement errors on the estimators is assessed.     

Sparse estimation in functional linear regression

As a useful tool in functional data analysis, the functional linear regression model has become increasingly common and been studied extensively in recent years. In this paper, we consider a sparse functional linear regression model which is generated by a finite number of basis functions in an expansion of the coefficient function. In this model, we do not specify how many and which basis functions enter the model, thus it is not like a typical parametric model where predictor variables are pre-specified. We study a general framework that gives various procedures which are successful in identifying the basis functions that enter the model, and also estimating the resulting regression coefficients in one-step. We adopt the idea of variable selection in the linear regression setting where one adds a weighted L₁ penalty to the traditional least squares criterion. We show that the procedures in our general framework are consistent in the sense of selecting the model correctly, and that they enjoy the oracle property, meaning that the resulting estimators of the coefficient function have asymptotically the same properties as the oracle estimator which uses knowledge of the underlying model. We investigate and compare several methods within our general framework, via a simulation study. Also, we apply the methods to the Canadian weather data.     

Estimation of normal covariance matrices parametrized by irreducible symmetric cones under Stein's loss

In this paper the problem of estimating a covariance matrix parametrized by an irreducible symmetric cone in a decision-theoretic set-up is considered. By making use of some results developed in a theory of finite-dimensional Euclidean simple Jordan algebras, Bartlett's decomposition and an unbiased risk estimate formula for a general family of Wishart distributions on the irreducible symmetric cone are derived; these results lead to an extension of Stein's general technique for derivation of minimax estimators for a real normal covariance matrix. Specification of the results to the multivariate normal models with covariances which are parametrized by complex, quaternion, and Lorentz types gives minimax estimators for each model.     

On Hsu's theorem in multivariate regression

The paper deals with optimal quadratic unbiased estimation of the unknown dispersion matrix in multivariate regression models without assuming normality of the errors. We show that Hsu's theorem for univariate regression models continues to multivariate models with no additional assumptions. Furthermore optimal quadratic plus linear estimating functions for regression coefficients are considered, and we investigate whether the ordinary linear estimates are the best. This leads to a new theorem which is similar to that of Hsu.     

Modifying estimators of ordered positive parameters under the Stein loss

This paper treats the problem of estimating positive parameters restricted to a polyhedral convex cone which includes typical order restrictions, such as simple order, tree order and umbrella order restrictions. In this paper, two methods are used to show the improvement of order-preserving estimators over crude non-order-preserving estimators without any assumption on underlying distributions. One is to use Fenchel’s duality theorem, and then the superiority of the isotonic regression estimator is established under the general restriction to polyhedral convex cones. The use of the Abel identity is the other method, and we can derive a class of improved estimators which includes order-statistics-based estimators in the typical order restrictions. When the underlying distributions are scale families, the unbiased estimators and their order-restricted estimators are shown to be minimax. The minimaxity of the restrictedly generalized Bayes estimator against the prior over the restricted space is also demonstrated in the two dimensional case. Finally, some examples and multivariate extensions are given.     

Modelling total tail dependence along diagonals

An approach to modelling total tail dependence beyond the main diagonals is proposed. The concept introduced combines the principal and minor diagonals to describe total extreme dependence. A framework is introduced for the measurement of total tail dependence under model mixture. Illustrations are presented using empirical data on stock market indices and exchange rates. An extension is provided to the multivariate case and total tail dependence is considered for model mixtures.     

Affine-invariant aligned rank tests for the multivariate general linear model with VARMA errors

We develop optimal rank-based procedures for testing affine-invariant linear hypotheses on the parameters of a multivariate general linear model with elliptical VARMA errors. We propose a class of optimal procedures that are based either on residual (pseudo-)Mahalanobis signs and ranks, or on absolute interdirections and lift-interdirection ranks, i.e., on hyperplane-based signs and ranks. The Mahalanobis versions of these procedures are strictly affine-invariant, while the hyperplane-based ones are asymptotically affine-invariant. Both versions generalize the univariate signed rank procedures proposed by Hallin and Puri (J. Multivar. Anal. 50 (1994) 175), and are locally asymptotically most stringent under correctly specified radial densities. Their AREs with respect to Gaussian procedures are shown to be convex linear combinations of the AREs obtained in Hallin and Paindaveine (Ann. Statist. 30 (2002) 1103; Bernoulli 8 (2002) 787) for the pure location and purely serial models, respectively. The resulting test statistics are provided under closed form for several important particular cases, including multivariate Durbin-Watson tests, VARMA order identification tests, etc. The key technical result is a multivariate asymptotic linearity result proved in Hallin and Paindaveine (Asymptotic linearity of serial and nonserial multivariate signed rank statistics, submitted).     

Who's afraid of reduced-rank parameterizations of multivariate models? Theory and example

Reduced-rank restrictions can add useful parsimony to coefficient matrices of multivariate models, but their use is limited by the daunting complexity of the methods and their theory. The present work takes the easy road, focusing on unifying themes and simplified methods. For Gaussian and non-Gaussian (GLM, GAM, mixed normal, etc.) multivariate models, the present work gives a unified, explicit theory for the general asymptotic (normal) distribution of maximum likelihood estimators (MLE). MLE can be complex and computationally hard, but we show a strong asymptotic equivalence between MLE and a relatively simple minimum (Mahalanobis) distance estimator. The latter method yields particularly simple tests of rank, and we describe its asymptotic behavior in detail. We also examine the method's performance in simulation and via analytical and empirical examples.     

Portfolio value at risk based on independent component analysis

Risk management technology applied to high-dimensional portfolios needs simple and fast methods for calculation of value at risk (VaR). The multivariate normal framework provides a simple off-the-shelf methodology but lacks the heavy-tailed distributional properties that are observed in data. A principle component-based method (tied closely to the elliptical structure of the distribution) is therefore expected to be unsatisfactory. Here, we propose and analyze a technology that is based on independent component analysis (ICA). We study the proposed ICVaR methodology in an extensive simulation study and apply it to a high-dimensional portfolio situation. Our analysis yields very accurate VaRs.     

Estimation for a marginal generalized single-index longitudinal model

In this paper, we suggest an estimating equations based approach to study a general single-index model with a given out-layer link for longitudinal data and treat the classical one as its special case. Within a wide range of bandwidths which is for estimating the inner-layer nonparametric link, the root-n consistency of the estimator of the index can be proved. The estimation efficiency can be achieved even when there is an infinite-dimensional nuisance parameter to be estimated. The performance of the new method is assessed through the comparison with other existing methods and through an application to an epileptic seizure study.     

Restricted expected multivariate least squares

A new approach of estimating parameters in multivariate models is introduced. A fitting function will be used. The idea is to estimate parameters so that the fitting function equals or will be close to its expected value. The function will be decomposed into two parts. From one part, which will be independent of the mean parameters, the dispersion matrix is estimated. This estimator is inserted in the second part which then yields the estimators of the mean parameters. The Growth Curve model, extended Growth Curve model and a multivariate variance components model will illustrate the approach.     

Methods for improvement in estimation of a normal mean matrix

This paper is concerned with the problem of estimating a matrix of means in multivariate normal distributions with an unknown covariance matrix under invariant quadratic loss. It is first shown that the modified Efron-Morris estimator is characterized as a certain empirical Bayes estimator. This estimator modifies the crude Efron-Morris estimator by adding a scalar shrinkage term. It is next shown that the idea of this modification provides a general method for improvement of estimators, which results in the further improvement on several minimax estimators. As a new method for improvement, an adaptive combination of the modified Stein and the James-Stein estimators is also proposed and is shown to be minimax. Through Monte Carlo studies of the risk behaviors, it is numerically shown that the proposed, combined estimator inherits the nice risk properties of both individual estimators and thus it has a very favorable risk behavior in a small sample case. Finally, the application to a two-way layout MANOVA model with interactions is discussed.     

Empirical Bayes predictive densities for high-dimensional normal models

This paper addresses the problem of estimating the density of a future outcome from a multivariate normal model. We propose a class of empirical Bayes predictive densities and evaluate their performances under the Kullback–Leibler (KL) divergence. We show that these empirical Bayes predictive densities dominate the Bayesian predictive density under the uniform prior and thus are minimax under some general conditions. We also establish the asymptotic optimality of these empirical Bayes predictive densities in infinite-dimensional parameter spaces through an oracle inequality.     

Estimates of MM type for the multivariate linear model

We propose a class of robust estimates for multivariate linear models. Based on the approach of MM-estimation (Yohai 1987, [24]), we estimate the regression coefficients and the covariance matrix of the errors simultaneously. These estimates have both a high breakdown point and high asymptotic efficiency under Gaussian errors. We prove consistency and asymptotic normality assuming errors with an elliptical distribution. We describe an iterative algorithm for the numerical calculation of these estimates. The advantages of the proposed estimates over their competitors are demonstrated through both simulated and real data.     

Bayesian analysis of multivariate t linear mixed models using a combination of IBF and Gibbs samplers

The multivariate linear mixed model (MLMM) has become the most widely used tool for analyzing multi-outcome longitudinal data. Although it offers great flexibility for modeling the between- and within-subject correlation among multi-outcome repeated measures, the underlying normality assumption is vulnerable to potential atypical observations. We present a fully Bayesian approach to the multivariate t linear mixed model (MtLMM), which is a robust extension of MLMM with the random effects and errors jointly distributed as a multivariate t distribution. Owing to the introduction of too many hidden variables in the model, the conventional Markov chain Monte Carlo (MCMC) method may converge painfully slowly and thus fails to provide valid inference. To alleviate this problem, a computationally efficient inverse Bayes formulas (IBF) sampler coupled with the Gibbs scheme, called the IBF-Gibbs sampler, is developed and shown to be effective in drawing samples from the target distributions. The issues related to model determination and Bayesian predictive inference for future values are also investigated. The proposed methodologies are illustrated with a real example from an AIDS clinical trial and a careful simulation study.     

Asymptotic Bayesian structure learning using graph supports for Gaussian graphical models

The theory of Gaussian graphical models is a powerful tool for independence analysis between continuous variables. In this framework, various methods have been conceived to infer independence relations from data samples. However, most of them result in stepwise, deterministic, descent algorithms that are inadequate for solving this issue. More recent developments have focused on stochastic procedures, yet they all base their research on strong a priori knowledge and are unable to perform model selection among the set of all possible models. Moreover, convergence of the corresponding algorithms is slow, precluding applications on a large scale. In this paper, we propose a novel Bayesian strategy to deal with structure learning. Relating graphs to their supports, we convert the problem of model selection into that of parameter estimation. Use of non-informative priors and asymptotic results yield a posterior probability for independence graph supports in closed form. Gibbs sampling is then applied to approximate the full joint posterior density. We finally give three examples of structure learning, one from synthetic data, and the two others from real data.     

Likelihood ratio tests of correlated multivariate samples

We develop methods to compare multiple multivariate normally distributed samples which may be correlated. The methods are new in the context that no assumption is made about the correlations among the samples. Three types of null hypotheses are considered: equality of mean vectors, homogeneity of covariance matrices, and equality of both mean vectors and covariance matrices. We demonstrate that the likelihood ratio test statistics have finite-sample distributions that are functions of two independent Wishart variables and dependent on the covariance matrix of the combined multiple populations. Asymptotic calculations show that the likelihood ratio test statistics converge in distribution to central Chi-squared distributions under the null hypotheses regardless of how the populations are correlated. Following these theoretical findings, we propose a resampling procedure for the implementation of the likelihood ratio tests in which no restrictive assumption is imposed on the structures of the covariance matrices. The empirical size and power of the test procedure are investigated for various sample sizes via simulations. Two examples are provided for illustration. The results show good performance of the methods in terms of test validity and power.     

A likelihood ratio test for separability of covariances

We propose a formal test of separability of covariance models based on a likelihood ratio statistic. The test is developed in the context of multivariate repeated measures (for example, several variables measured at multiple times on many subjects), but can also apply to a replicated spatio-temporal process and to problems in meteorology, where horizontal and vertical covariances are often assumed to be separable. Separable models are a common way to model spatio-temporal covariances because of the computational benefits resulting from the joint space-time covariance being factored into the product of a covariance function that depends only on space and a covariance function that depends only on time. We show that when the null hypothesis of separability holds, the distribution of the test statistic does not depend on the type of separable model. Thus, it is possible to develop reference distributions of the test statistic under the null hypothesis. These distributions are used to evaluate the power of the test for certain nonseparable models. The test does not require second-order stationarity, isotropy, or specification of a covariance model. We apply the test to a multivariate repeated measures problem.     

Time-varying coefficient estimation in differential equation models with noisy time-varying covariates

We study the problem of estimating time-varying coefficients in ordinary differential equations. Current theory only applies to the case when the associated state variables are observed without measurement errors as presented in Chen and Wu (2008) [4] and [5]. The difficulty arises from the quadratic functional of observations that one needs to deal with instead of the linear functional that appears when state variables contain no measurement errors. We derive the asymptotic bias and variance for the previously proposed two-step estimators using quadratic regression functional theory.