Hypothesis tests for high-dimensional covariance structures

We consider hypothesis testing for high-dimensional covariance structures in which the covariance matrix is a (i) scaled identity matrix, (ii) diagonal matrix, or (iii) intraclass covariance matrix. Our purpose is to systematically establish a nonparametric approach for testing the high-dimensional covariance structures (i)–(iii). We produce a new common test statistic for each covariance structure and show that the test statistic is an unbiased estimator of its corresponding test parameter. We prove that the test statistic establishes the asymptotic normality. We propose a new test procedure for (i)–(iii) and evaluate its asymptotic size and power theoretically when both the dimension and sample size increase. We investigate the performance of the proposed test procedure in simulations. As an application of testing the covariance structures, we give a test procedure to identify an eigenvector. Finally, we demonstrate the proposed test procedure by using a microarray data set.

     

Covariance estimation under spatial dependence

Correlated multivariate processes have a dependence structure which must be taken into account when estimating the covariance matrix. The natural estimator of the covariance matrix is introduced and is shown that to be biased under the dependence structure. This bias is studied under two different asymptotic models, namely increasing the domain by increasing the number of observations, and increasing the number of observations in the fixed domain. Using the first asymptotic model, we quantify the convergence rate of the bias and of the covariance between the components of the estimated covariance matrix. The second asymptotic model serves to derive a fast and accurate bias correction. As shown, under mild hypotheses, the asymptotic normality of the estimated covariance matrix holds and can be used to test whether the bias is significant, for example, in the sense that the eigenvectors of the estimated and true covariance matrices are significantly different.     

HAC estimation and strong linearity testing in weak ARMA models

In the framework of ARMA models, we consider testing the reliability of the standard asymptotic covariance matrix (ACM) of the least-squares estimator. The standard formula for this ACM is derived under the assumption that the errors are independent and identically distributed, and is in general invalid when the errors are only uncorrelated. The test statistic is based on the difference between a conventional estimator of the ACM of the least-squares estimator of the ARMA coefficients and its robust HAC-type version. The asymptotic distribution of the HAC estimator is established under the null hypothesis of independence, and under a large class of alternatives. The asymptotic distribution of the proposed statistic is shown to be a standard χ² under the null, and a noncentral χ² under the alternatives. The choice of the HAC estimator is discussed through asymptotic power comparisons. The finite sample properties of the test are analyzed via Monte Carlo simulation.     

Robustifying principal component analysis with spatial sign vectors

In this paper, we apply orthogonally equivariant spatial sign covariance matrices as well as their affine equivariant counterparts in principal component analysis. The influence functions and asymptotic covariance matrices of eigenvectors based on robust covariance estimators are derived in order to compare the robustness and efficiency properties. We show in particular that the estimators that use pairwise differences of the observed data have very good efficiency properties, providing practical robust alternatives to classical sample covariance matrix based methods.     

The asymptotic distribution of a goodness of fit statistic for factorial invariance

Suppose that random factor models with k factors are assumed to hold for m, p-variate populations. A model for factorial invariance has been proposed wherein the covariance or correlation matrices can be written as Σ_i = LC_iL′ + σ_i²I, where C_i is the covariance matrix of factor variables and L is a common factor loading matrix, i = 1,…, m. Also a goodness of fit statistic has been proposed for this model. The asymptotic distribution of this statistic is shown to be that of a quadratic form in normal variables. An approximation to this distribution is given and thus a test for goodness of fit is derived. The problem of dimension is considered and a numerical example is given to illustrate the results.     

On the distributions of some test criteria for a covariance matrix under local alternatives and bootstrap approximations

The asymptotic distribution of some test criteria for a covariance matrix are derived under local alternatives. Except for the existence of some higher moments, no assumption as to the form of the distribution function is made. As an illustration, a case of t distribution included normal model is considered and the power of the likelihood ratio test and Nagao's test for sphericity, as described in Srivastava and Khatri and Anderson, is computed. Also, the power is computed using the bootstrap method. In the case of t distribution, the bootstrap approximation does not appear to be as good as the one obtained by the asymptotic expansion method.     

Testing equality between several populations covariance operators

In many situations, when dealing with several populations, equality of the covariance operators is assumed. An important issue is to study whether this assumption holds before making other inferences. In this paper, we develop a test for comparing covariance operators of several functional data samples. The proposed test is based on the Hilbert–Schmidt norm of the difference between estimated covariance operators. In particular, when dealing with two populations, the test statistic is just the squared norm of the difference between the two covariance operators estimators. The asymptotic behaviour of the test statistic under both the null hypothesis and local alternatives is obtained. The computation of the quantiles of the null asymptotic distribution is not feasible in practice. To overcome this problem, a bootstrap procedure is considered. The performance of the test statistic for small sample sizes is illustrated through a Monte Carlo study and on a real data set.     

Testing for elliptical symmetry in covariance-matrix-based analyses

Many normal-theory test procedures for covariance matrices remain valid outside the family of normal distributions if the matrix of fourth-order moments has structure similar to that of a normal distribution. In particular, for elliptical distributions this matrix of fourth-order moments is a scalar multiple of that for the normal, and for this reason many normal-theory statistics can be adjusted by a scalar multiple so as to retain their asymptotic distributional properties across elliptical distributions. For these analyses, a test for the validity of these scalar-adjusted normal-theory procedures can be viewed as a test on the structure of the matrix of fourth-order moments. In this paper, we develop a Wald statistic for conducting such a test.     

Growth Curve Model with Hierarchical Within-Individuals Design Matrices

This paper deals with some inferential problems under an extended growth curve model with several hierarchical within-individuals design matrices. The model includes the one whose mean structure consists of polynomial growth curves with different degrees. First we consider the case when the covariance matrix is unknown positive definite. We derive a LR test for examining the hierarchical structure for within individuals design matrices and a model selection criterion. Next we consider the case when a random coefficients covariance structure is assumed, under certain assumption of between-individual design matrices. Similar inferential problems are also considered. The dental measurement data (see, e.g., Potthoff and Roy (1964, Biometrika, 51, 313-326)) is reexamined, based on extended growth curve models.     

Asymptotic expansion of the null distribution of test statistic for linear hypothesis in nonnormal linear model

This paper is concerned with the null distribution of test statistic T for testing a linear hypothesis in a linear model without assuming normal errors. The test statistic includes typical ANOVA test statistics. It is known that the null distribution of T converges to χ² when the sample size n is large under an adequate condition of the design matrix. We extend this result by obtaining an asymptotic expansion under general condition. Next, asymptotic expansions of one- and two-way test statistics are obtained by using this general one. Numerical accuracies are studied for some approximations of percent points and actual test sizes of T for two-way ANOVA test case based on the limiting distribution and an asymptotic expansion.     

Principal Component Analysis from the Multivariate Familial Correlation Matrix

This paper considers principal component analysis (PCA) in familial models, where the number of siblings can differ among families. S. Konishi and C. R. Rao (1992, Biometrika79, 631–641) used the unified estimator of S. Konishi and C. G. Khatri (1990, Ann. Inst. Statist. Math.42, 561–580) to develop a PCA derived from the covariance matrix. However, because of the lack of invariance to componentwise change of scale, an analysis based on the correlation matrix is often preferred. The asymptotic distribution of the estimated eigenvalues and eigenvectors of the correlation matrix are derived under elliptical sampling. A Monte Carlo simulation shows the usefulness of the asymptotic expressions for samples as small as N=25 families.     

A new test for sphericity of the covariance matrix for high dimensional data

In this paper we propose a new test procedure for sphericity of the covariance matrix when the dimensionality, p, exceeds that of the sample size, N=n+1. Under the assumptions that (A) as p→∞ for i=1,…,16 and (B) p/n→c<∞ known as the concentration, a new statistic is developed utilizing the ratio of the fourth and second arithmetic means of the eigenvalues of the sample covariance matrix. The newly defined test has many desirable general asymptotic properties, such as normality and consistency when (n,p)→∞. Our simulation results show that the new test is comparable to, and in some cases more powerful than, the tests for sphericity in the current literature.     

A Quantile Goodness-of-Fit Test for Cauchy Distribution, Based on Extreme Order Statistics

A test statistic for testing goodness-of-fit of the Cauchy distribution is presented. It is a quadratic form of the first and of the last order statistic and its matrix is the inverse of the asymptotic covariance matrix of the quantile difference statistic. The distribution of the presented test statistic does not depend on the parameter of the sampled Cauchy distribution. The paper contains critical constants for this test statistic, obtained from 50,000 simulations for each sample size considered. Simulations show that the presented test statistic is for testing goodness-of-fit of the Cauchy distributions more powerful than the Anderson-Darling, Kolmogorov-Smirnov or the von Mises test statistic.     

A well-conditioned estimator for large-dimensional covariance matrices

Many applied problems require a covariance matrix estimator that is not only invertible, but also well-conditioned (that is, inverting it does not amplify estimation error). For large-dimensional covariance matrices, the usual estimator—the sample covariance matrix—is typically not well-conditioned and may not even be invertible. This paper introduces an estimator that is both well-conditioned and more accurate than the sample covariance matrix asymptotically. This estimator is distribution-free and has a simple explicit formula that is easy to compute and interpret. It is the asymptotically optimal convex linear combination of the sample covariance matrix with the identity matrix. Optimality is meant with respect to a quadratic loss function, asymptotically as the number of observations and the number of variables go to infinity together. Extensive Monte Carlo confirm that the asymptotic results tend to hold well in finite sample.     

Asymptotic distribution of the LR statistic for equality of the smallest eigenvalues in high-dimensional principal component analysis

This paper deals with the distribution of the LR statistic for testing the hypothesis that the smallest eigenvalues of a covariance matrix are equal. We derive an asymptotic null distribution of the LR statistic when the dimension p and the sample size N approach infinity, while the ratio p/N converging on a finite nonzero limit c(0,1). Numerical simulations revealed that our approximation is more accurate than the classical chi-square-type approximation as p increases in value.     

On the sensitivity of the one-sided test to covariance misspecification

Sensitivity analysis stands in contrast to diagnostic testing in that sensitivity analysis aims to answer the question of whether it matters that a nuisance parameter is non-zero, whereas a diagnostic test ascertains explicitly if the nuisance parameter is different from zero. In this paper, we introduce and derive the finite sample properties of a sensitivity statistic measuring the sensitivity of the t statistic to covariance misspecification. Unlike the earlier work by Banerjee and Magnus [A. Banerjee, J.R. Magnus, On the sensitivity of the usual t- and F-tests to covariance misspecification, Journal of Econometrics 95 (2000) 157–176] on the sensitivity of the F statistic, the theorems derived in the current paper hold under both the null and alternative hypotheses. Also, in contrast to Banerjee and Magnus’ [see the above cited reference] results on the F test, we find that the decision to accept the null using the OLS based one-sided t test is not necessarily robust against covariance misspecification and depends much on the underlying data matrix. Our results also indicate that autocorrelation does not necessarily weaken the power of the OLS based t test.     

Fixed Width Confidence Region for the Mean of a Multivariate Normal Distribution

Srivastava gave an asymptotically efficient and consistent sequential procedure to obtain a fixed-width confidence region for the mean vector of any p-dimensional random vector with finite second moments. For normally distributed random vectors, Srivastava and Bhargava showed that the specified coverage probability is attained independent of the width, the mean vector, and the covariance matrix by taking a finite number of observations over and above T prescribed by the sequential rule. However, the problem of showing that E(T−n₀) is bounded, where n₀ is the sample size required if the covariance matrix were known, has not been available. In this paper, we not only show that it is bounded but obtain sharper estimates of E(T) on the lines of Woodroofe. We also give an asymptotic expansion of the coverage probability. Similar results have recently been obtained by Nagao under the assumption that the covariance matrix Σ=∑^k_i=1 σ_iA_i and ∑^k_i=1 A_i=I, where A_i's are known matrices. We obtain the results of this paper under the sole assumption that the largest latent root of Σ is simple.     

Relative efficiencies of estimates using patterned covariances or correlations in the multivariate normal estimation problem

Summary The relative efficiency of maximum likelihood estimates is studied when taking advantage of underlying linear patterns in the covariances or correlations when estimating covariance matrices. We compare the variances of estimates of the covariance matrix obtained under two nested patterns with the assumption that the more restricted pattern is the true state. Formulas for the asymptotic variances are given which are exact for linear covariance patterns when explicit maximum likelihood estimates exist. Several specific examples are given using complete symmetry, circular symmetry and general covariance patterns as well as an example involving a covariance matrix with a linear pattern in the correlations.     

Sphericity Test for High Dimensional Data Based on Random Matrix Theory

In this article we study test of sphericity for high-dimensional covariance matrix in the general population based on random matrix theory. When the sample size is less than data dimension, the classical likelihood ratio test has poor performance for test of sphericity. Thus, we propose a new statistic for test of sphericity by using the higher moments of spectral distribution of the sample covariance matrix, and derive the asymptotic distribution of the statistic under the null hypothesis. Simulation results show that the proposed statistics can effectively improve the power of the test of sphericity for high dimensional data, and have especially significant effects for Spiked model, on the basis of controlling the type-one error probability.     

Testing the covariance function of stationary Gaussian random fields

In many problems, a specific function like h(⋅) is considered as the covariance function. Based on the asymptotic distribution of the periodogram and Euler characteristic, three methods are introduced to test the equality of the covariance function with h(⋅). Our analyses prove the accuracy of the power and scaling laws for the covariance function of metal surfaces.