On testing for clusters using the sample covariance

This paper analyzes the problem of using the sample covariance matrix to detect the presence of clustering in p-variate data in the special case when the component covariance matrices are known up to a constant multiplier. For the case of testing one population against a mixture of two populations, tests are derived and shown to be optimal in a certain sense. Some of their distribution properties are derived exactly. Some remarks on the extensions of these tests to mixtures of k ≤ p populations are included. The paper is essentially a formal treatment (in a special case) of some well-known procedures. The methods used in deriving the distribution properties are applicable to a variety of other situations involving mixtures.     

On the principal components of sample covariance matrices

We introduce a class of \(M \times M\) sample covariance matrices \({\mathcal {Q}}\) which subsumes and generalizes several previous models. The associated population covariance matrix \(\Sigma = \mathbb {E}{\mathcal {Q}}\) is assumed to differ from the identity by a matrix of bounded rank. All quantities except the rank of \(\Sigma - I_M\) may depend on \(M\) in an arbitrary fashion. We investigate the principal components, i.e. the top eigenvalues and eigenvectors, of \({\mathcal {Q}}\). We derive precise large deviation estimates on the generalized components \(\langle {\mathbf{{w}}} , {\varvec{\xi }_i}\rangle \) of the outlier and non-outlier eigenvectors \(\varvec{\xi }_i\). Our results also hold near the so-called BBP transition, where outliers are created or annihilated, and for degenerate or near-degenerate outliers. We believe the obtained rates of convergence to be optimal. In addition, we derive the asymptotic distribution of the generalized components of the non-outlier eigenvectors. A novel observation arising from our results is that, unlike the eigenvalues, the eigenvectors of the principal components contain information about the subcritical spikes of \(\Sigma \). The proofs use several results on the eigenvalues and eigenvectors of the uncorrelated matrix \({\mathcal {Q}}\), satisfying \(\mathbb {E}{\mathcal {Q}} = I_M\), as input: the isotropic local Marchenko–Pastur law established in Bloemendal et al. (Electron J Probab 19:1–53, 2014), level repulsion, and quantum unique ergodicity of the eigenvectors. The latter is a special case of a new universality result for the joint eigenvalue–eigenvector distribution.     

Nonnull distributions of some statistics associated with testing for the equality of two covariance matrices

The nonnull distribution of some statistics, used for testing Σ₁ = Σ₂ are obtained as mixtures of incomplete beta functions as well as mixtures of incomplete gamma functions. The introduction of the convergence factors and certain recurrence relations are useful in the computation of the power of the tests as well as computation of exact percentage points for tests of significance.     

On an optimum test of the equality of two covariance matrices

Let X: p × 1, Y: p × 1 be independently and normally distributed p-vectors with unknown means ₁, ₂ and unknown covariance matrices ₁, ₂ (>0) respectively. We shall show that Pillai's test, which is locally best invariant, is locally minimax for testing H ₀: ₁=₂ against the alternative H ₁: % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% GaaeiDaiaabkhacaqGOaWaaabmaeaadaaeqaqaaiabgkHiTiaadMea% caGGPaGaaiiiaiabg2da9iaacccacqaHdpWCcaGGGaGaeyOpa4Jaai% iiaiaaicdaaSqaaiaaigdaaeqaniabggHiLdaaleaacaqGYaaabaGa% aeylaiaabgdaa0GaeyyeIuoaaaa!4E3F!\[{\rm{tr(}}\sum\nolimits_{\rm{2}}^{{\rm{ - 1}}} {\sum\nolimits_1 { - I) = \sigma > 0} }\]as 0. However this test is not of type D among G-invariant tests.Research supported by the Canadian N.S.E.R.C. Grant.     

On the quadratic estimation of covariance matrices in multivariate linear models

This paper investigates the estimation of covariance matrices in multivariate mixed models. Some sufficient conditions are derived for a multivariate quadratic form and a linear combination of multivariate quadratic forms to be the BQUE (quadratic unbiased and severally minimum varianced) estimators of its expectations.     

On the limiting spectral distribution of the covariance matrices of time-lagged processes

We consider two continuous-time Gaussian processes, one being partially correlated to a time-lagged version of the other. We first give the limiting spectral distribution for the covariance matrices of the increments of the processes when the span between two observations tends to zero. Then, we derive the limiting distribution of the eigenvalues of the sample covariance matrices. This result is obtained when the number of paths of the processes is asymptotically proportional to the number of observations for each single path. As an application, we use the second moment of this distribution together with auxiliary volatility and correlation estimates to construct an adaptive estimator of the time lag between the two processes. Finally, we provide an asymptotic theory for our estimation procedure.     

A well-conditioned estimator for large-dimensional covariance matrices

Many applied problems require a covariance matrix estimator that is not only invertible, but also well-conditioned (that is, inverting it does not amplify estimation error). For large-dimensional covariance matrices, the usual estimator—the sample covariance matrix—is typically not well-conditioned and may not even be invertible. This paper introduces an estimator that is both well-conditioned and more accurate than the sample covariance matrix asymptotically. This estimator is distribution-free and has a simple explicit formula that is easy to compute and interpret. It is the asymptotically optimal convex linear combination of the sample covariance matrix with the identity matrix. Optimality is meant with respect to a quadratic loss function, asymptotically as the number of observations and the number of variables go to infinity together. Extensive Monte Carlo confirm that the asymptotic results tend to hold well in finite sample.     

Canonical spectral equation for empirical covariance matrices

We study asymptotic properties of normalized spectral functions of empirical covariance matrices in the case of a nonnormal population. It is shown that the Stieltjes transforms of such functions satisfy a socalled canonical spectral equation.Translated from Ukrainskii Matematicheskii Zhurnal, Vol. 47, No. 9, pp. 1176–1189, September, 1995.     

Proportionality of k covariance matrices

This article discusses maximum likelihood estimation of proportional covariance matrices under normality assumptions. An algorithm for solving the likelihood equations and the likelihood ratio statistic for testing the hypothesis of proportionality are given. The method is illustrated by a numerical example.     

Partial estimation of covariance matrices

A classical approach to accurately estimating the covariance matrix Σ of a p-variate normal distribution is to draw a sample of size n > p and form a sample covariance matrix. However, many modern applications operate with much smaller sample sizes, thus calling for estimation guarantees in the regime ${n \ll p}$ . We show that a sample of size n = O(m log⁶ p) is sufficient to accurately estimate in operator norm an arbitrary symmetric part of Σ consisting of m ≤ n nonzero entries per row. This follows from a general result on estimating Hadamard products M · Σ, where M is an arbitrary symmetric matrix.     

Two step-down tests for equality of covariance matrices

The classical problem of testing the equality of the covariance matrices from k ? 2 p-dimensional normal populations is reexamined. The likelihood ratio (LR) statistic, also called Bartlett’s statistic, can be decomposed in two ways, corresponding to two distinct component-wise decompositions of the null hypothesis in terms of the covariance matrices or precision matrices, respectively. The factors of the LR statistic that appear in these two decompositions can be interpreted as conditional and unconditional LR statistics for the component-wise null hypotheses, and their mutual independence under the null hypothesis allows the determination of the overall significance level.     

On a model selection problem from high-dimensional sample covariance matrices

Modern random matrix theory indicates that when the population size p is not negligible with respect to the sample size n, the sample covariance matrices demonstrate significant deviations from the population covariance matrices. In order to recover the characteristics of the population covariance matrices from the observed sample covariance matrices, several recent solutions are proposed when the order of the underlying population spectral distribution is known. In this paper, we deal with the underlying order selection problem and propose a solution based on the cross-validation principle. We prove the consistency of the proposed procedure.     

On the correlation functions of the characteristic polynomials of the hermitian sample covariance matrices

We consider asymptotic behavior of the correlation functions of the characteristic polynomials of the hermitian sample covariance matrices ${H_n=n^{-1}A_{m,n}^* A_{m,n}}$ , where A _m,n is a m × n complex random matrix with independent and identically distributed entries ${\mathfrak{R}a_{\alpha j}}$ and ${\mathfrak{I}a_{\alpha j}}$ . We show that for the correlation function of any even order the asymptotic behavior in the bulk and at the edge of the spectrum coincides with those for the Gaussian Unitary Ensemble up to a factor, depending only on the fourth moment of the common probability law of entries ${\mathfrak{R}a_{\alpha j}}$ , ${\mathfrak{I}a_{\alpha j}}$ , i.e., the higher moments do not contribute to the above limit.     

Spectra of infinite-dimensional sample covariance matrices

We study spectral functions of infinite-dimensional random Gram matrices of the form RR^T, where R is a rectangular matrix with an infinite number of rows and with the number of columns N → ∞, and the spectral functions of infinite sample covariance matrices calculated for samples of volume N → ∞ under conditions analogous to the Kolmogorov asymptotic conditions. We assume that the traces d of the expectations of these matrices increase with the number N such that the ratio d/N tends to a constant. We find the limiting nonlinear equations relating the spectral functions of random and nonrandom matrices and establish the asymptotic expression for the resolvent of random matrices. __________ Translated from Teoreticheskaya i Matematicheskaya Fizika, Vol. 148, No. 2, pp. 309–322, August, 2006.     

Optimal rates of convergence for estimating Toeplitz covariance matrices

Toeplitz covariance matrices are used in the analysis of stationary stochastic processes and a wide range of applications including radar imaging, target detection, speech recognition, and communications systems. In this paper, we consider optimal estimation of large Toeplitz covariance matrices and establish the minimax rate of convergence for two commonly used parameter spaces under the spectral norm. The properties of the tapering and banding estimators are studied in detail and are used to obtain the minimax upper bound. The results also reveal a fundamental difference between the tapering and banding estimators over certain parameter spaces. The minimax lower bound is derived through a novel construction of a more informative experiment for which the minimax lower bound is obtained through an equivalent Gaussian scale model and through a careful selection of a finite collection of least favorable parameters. In addition, optimal rate of convergence for estimating the inverse of a Toeplitz covariance matrix is also established.