Testing the equality of several covariance matrices with fewer observations than the dimension

For normally distributed data from the k populations with m×m covariance matrices Σ₁,…,Σ_k, we test the hypothesis H:Σ₁=?=Σ_k vs the alternative A≠H when the number of observations N_i, i=1,…,k from each population are less than or equal to the dimension m, N_i≤m, i=1,…,k. Two tests are proposed and compared with two other tests proposed in the literature. These tests, however, do not require that N_i≤m, and thus can be used in all situations, including when the likelihood ratio test is available. The asymptotic distributions of the test statistics are given, and the power compared by simulations with other test statistics proposed in the literature. The proposed tests perform well and better in several cases than the other two tests available in the literature.     

On the distribution of the ratio of the largest eigenvalue to the trace of a Wishart matrix

The ratio of the largest eigenvalue divided by the trace of a p×p random Wishart matrix with n degrees of freedom and an identity covariance matrix plays an important role in various hypothesis testing problems, both in statistics and in signal processing. In this paper we derive an approximate explicit expression for the distribution of this ratio, by considering the joint limit as both p,n→∞ with p/n→c. Our analysis reveals that even though asymptotically in this limit the ratio follows a Tracy-Widom (TW) distribution, one of the leading error terms depends on the second derivative of the TW distribution, and is non-negligible for practical values of p, in particular for determining tail probabilities. We thus propose to explicitly include this term in the approximate distribution for the ratio. We illustrate empirically using simulations that adding this term to the TW distribution yields a quite accurate expression to the empirical distribution of the ratio, even for small values of p,n.     

Wishart distributions associated with matrix quadratic forms

For a normally distributed random matrix Y with mean zero and general covariance matrix Σ_Y and for a symmetric matrix W, necessary and sufficient conditions are derived for the Wishartness of Y′WY.     

The asymptotic distribution of a goodness of fit statistic for factorial invariance

Suppose that random factor models with k factors are assumed to hold for m, p-variate populations. A model for factorial invariance has been proposed wherein the covariance or correlation matrices can be written as Σ_i = LC_iL′ + σ_i²I, where C_i is the covariance matrix of factor variables and L is a common factor loading matrix, i = 1,…, m. Also a goodness of fit statistic has been proposed for this model. The asymptotic distribution of this statistic is shown to be that of a quadratic form in normal variables. An approximation to this distribution is given and thus a test for goodness of fit is derived. The problem of dimension is considered and a numerical example is given to illustrate the results.     

Unbiased invariant minimum norm estimation in generalized growth curve model

This paper considers the generalized growth curve model subject to R(X_m)⊆R(X_m-1)⊆?⊆R(X₁), where B_i are the matrices of unknown regression coefficients, X_i,Z_i and U are known covariate matrices, i=1,2,…,m, and E splits into a number of independently and identically distributed subvectors with mean zero and unknown covariance matrix Σ. An unbiased invariant minimum norm quadratic estimator (MINQE(U,I)) of tr(CΣ) is derived and the conditions for its optimality under the minimum variance criterion are investigated. The necessary and sufficient conditions for MINQE(U,I) of tr(CΣ) to be a uniformly minimum variance invariant quadratic unbiased estimator (UMVIQUE) are obtained. An unbiased invariant minimum norm quadratic plus linear estimator (MINQLE(U,I)) of is also given. To compare with the existing maximum likelihood estimator (MLE) of tr(CΣ), we conduct some simulation studies which show that our proposed estimator performs very well.     

Multivariate analysis of variance with fewer observations than the dimension

In this article, we consider the problem of testing a linear hypothesis in a multivariate linear regression model which includes the case of testing the equality of mean vectors of several multivariate normal populations with common covariance matrix Σ, the so-called multivariate analysis of variance or MANOVA problem. However, we have fewer observations than the dimension of the random vectors. Two tests are proposed and their asymptotic distributions under the hypothesis as well as under the alternatives are given under some mild conditions. A theoretical comparison of these powers is made.     

Equivalent conditions for noncentral generalized Laplacianness and independence of matrix quadratic forms

Let Y be an n×p multivariate normal random matrix with general covariance Σ_Y and W be a symmetric matrix. In the present article, the property that a matrix quadratic form Y^′WY is distributed as a difference of two independent (noncentral) Wishart random matrices is called the (noncentral) generalized Laplacianness (GL). Then a set of algebraic results are obtained which will give the necessary and sufficient conditions for the (noncentral) GL of a matrix quadratic form. Further, two extensions of Cochran’s theorem concerning the (noncentral) GL and independence of a family of matrix quadratic forms are developed.     

Estimation of multivariate normal covariance and precision matrices in a star-shape model with missing data

In this paper, we study the problem of estimating the covariance matrix Σ and the precision matrix Ω (the inverse of the covariance matrix) in a star-shape model with missing data. By considering a type of Cholesky decomposition of the precision matrix Ω=Ψ^′Ψ, where Ψ is a lower triangular matrix with positive diagonal elements, we get the MLEs of the covariance matrix and precision matrix and prove that both of them are biased. Based on the MLEs, unbiased estimators of the covariance matrix and precision matrix are obtained. A special group G, which is a subgroup of the group consisting all lower triangular matrices, is introduced. By choosing the left invariant Haar measure on G as a prior, we obtain the closed forms of the best equivariant estimates of Ω under any of the Stein loss, the entropy loss, and the symmetric loss. Consequently, the MLE of the precision matrix (covariance matrix) is inadmissible under any of the above three loss functions. Some simulation results are given for illustration.     

On the structure of the Wishart distribution

In this paper it is shown that every nonnegative definite symmetric random matrix with independent diagonal elements and at least one nondegenerate nondiagonal element has a noninfinitely divisible distribution. Using this result it is established that every Wishart distribution W_p(k, Σ, M) with both p and rank (Σ) ≥ 2 is noninfinitely divisible. The paper also establishes that any Wishart matrix having distribution W_p(k, Σ, 0) has the joint distribution of its elements in the rth row and rth column to be infinitely divisible for every r = 1,2,…,p.     

Estimation of the mean vector of a multivariate normal distribution: subspace hypothesis

This paper considers the estimation of the mean vector θ of a p-variate normal distribution with unknown covariance matrix Σ when it is suspected that for a p×r known matrix B the hypothesis θ=Bη, η∈R^r may hold. We consider empirical Bayes estimators which includes (i) the unrestricted unbiased (UE) estimator, namely, the sample mean vector (ii) the restricted estimator (RE) which is obtained when the hypothesis θ=Bη holds (iii) the preliminary test estimator (PTE), (iv) the James-Stein estimator (JSE), and (v) the positive-rule Stein estimator (PRSE). The biases and the risks under the squared loss function are evaluated for all the five estimators and compared. The numerical computations show that PRSE is the best among all the five estimators even when the hypothesis θ=Bη is true.     

On testing for clusters using the sample covariance

This paper analyzes the problem of using the sample covariance matrix to detect the presence of clustering in p-variate data in the special case when the component covariance matrices are known up to a constant multiplier. For the case of testing one population against a mixture of two populations, tests are derived and shown to be optimal in a certain sense. Some of their distribution properties are derived exactly. Some remarks on the extensions of these tests to mixtures of k ≤ p populations are included. The paper is essentially a formal treatment (in a special case) of some well-known procedures. The methods used in deriving the distribution properties are applicable to a variety of other situations involving mixtures.     

Two testing problems relating the real and complex multivariate normal distributions

The first problem considered is that of testing for the reality of the covariance matrix of a p-dimensional complex normal distribution, while the second is that of testing that a 2p-dimensional real normal distribution has a p-dimensional complex structure. Both problems are reduced by invariance to their maximal invariant statistics, and the null and non-null distributions of these are obtained. Complete classes of unbiased, invariant tests are described for both problems, the locally most powerful invariant tests are obtained, and the admissibility of the likelihood ratio tests is established.     

Minimax and admissible minimax estimators of the mean of a multivariate normal distribution for unknown covariance matrix

Let X be a p-variate (p ≥ 3) vector normally distributed with mean μ and covariance Σ, and let A be a p × p random matrix distributed independent of X, according to the Wishart distribution W(n, Σ). For estimating μ, we consider estimators of the form δ = δ(X, A). We obtain families of Bayes, minimax and admissible minimax estimators with respect to the quadratic loss function (δ ? μ)′ Σ^?1(δ ? μ) where Σ is unknown. This paper extends previous results of the author [1], given for the case in which the covariance matrix of the distribution is of the form σ²I, where σ is known.     

Universality in complex Wishart ensembles for general covariance matrices with 2 distinct eigenvalues

We considered N×N Wishart ensembles in the class W_C(Σ_N,M) (complex Wishart matrices with M degrees of freedom and covariance matrix Σ_N) such that N₀ eigenvalues of Σ_N are 1 and N₁=N−N₀ of them are a. We studied the limit as M, N, N₀ and N₁ all go to infinity such that , and 0<c,β<1. In this case, the limiting eigenvalue density can either be supported on 1 or 2 disjoint intervals in R₊, and a phase transition occurs when the support changes from 1 interval to 2 intervals. By using the Riemann-Hilbert analysis, we have shown that when the phase transition occurs, the eigenvalue distribution is described by the Pearcey kernel near the critical point where the support splits.     

Characterization of Wishart-Laplace distributions via Jordan algebra homomorphisms

For a real, Hermitian, or quaternion normal random matrix Y with mean zero, necessary and sufficient conditions for a quadratic form Q(Y) to have a Wishart-Laplace distribution (the distribution of the difference of two independent central Wishart W_p(m_i,Σ) random matrices) are given in terms of a certain Jordan algebra homomorphism ρ. Further, it is shown that {Q_k(Y)} is independent Laplace-Wishart if and only if in addition to the aforementioned conditions, the images ρ_k(Σ⁺) of the Moore-Penrose inverse Σ⁺ of Σ are mutually orthogonal: ρ_k(Σ⁺)ρ_?(Σ⁺)=0 for k≠?.     

Robustness of A-optimal designs

Suppose that Y=(Y_i) is a normal random vector with mean Xb and covariance σ²I_n, where b is a p-dimensional vector (b_j),X=(X_ij) is an n×p matrix. A-optimal designs X are chosen from the traditional set D of A-optimal designs for ρ=0 such that X is still A-optimal in D when the components Y_i are dependent, i.e., for i≠i′, the covariance of Y_i,Y_i^′ is ρ with ρ≠0. Such designs depend on the sign of ρ. The general results are applied to X=(X_ij), where X_ij∈{-1,1}; this corresponds to a factorial design with -1,1 representing low level or high level respectively, or corresponds to a weighing design with -1,1 representing an object j with weight b_j being weighed on the left and right of a chemical balance respectively.     

On monotonicity of the modified likelihood ratio test for the equality of two covariances

For testing the hypothesis of equality of two covariances (Σ₁ and Σ₂) of two p-dimensional multivariate normal populations, it is shown that the power function of the modified likelihood ratio test increases as λ₁ increases from one and λ_r decreases from one where λ₁ > … > λ_r > 0 are the distinct characteristic roots of Σ₁Σ₂^?1, r ≤ p. As a by-product we get the unbiased result already established by Sugiura and Nagao (1968).     

Distribution and characteristic functions for correlated complex Wishart matrices

Let A(t) be a complex Wishart process defined in terms of the M×N complex Gaussian matrix X(t) by A(t)=X(t)X(t)^H. The covariance matrix of the columns of X(t) is Σ. If X(t), the underlying Gaussian process, is a correlated process over time, then we have dependence between samples of the Wishart process. In this paper, we study the joint statistics of the Wishart process at two points in time, t₁, t₂, where t₁<t₂. In particular, we derive the following results: the joint density of the elements of A(t₁), A(t₂), the joint density of the eigenvalues of Σ^-1A(t₁),Σ^-1A(t₂), the characteristic function of the elements of A(t₁), A(t₂), the characteristic function of the eigenvalues of Σ^-1A(t₁),Σ^-1A(t₂). In addition, we give the characteristic functions of the eigenvalues of a central and non-central complex Wishart, and some applications of the results in statistics, engineering and information theory are outlined.     

Error bounds for asymptotic expansions of Wilks’ lambda distribution

Let Λ=|S_e|/|S_e+S_h|, where S_h and S_e are independently distributed as Wishart distributions W_p(q,Σ) and W_p(n,Σ), respectively. Then Λ has Wilks’ lambda distribution Λ_p,q,n which appears as the distributions of various multivariate likelihood ratio tests. This paper is concerned with theoretical accuracy for asymptotic expansions of the distribution of T=-nlogΛ. We derive error bounds for the approximations. It is necessary to underline that our error bounds are given in explicit and computable forms.     

A new test for sphericity of the covariance matrix for high dimensional data

In this paper we propose a new test procedure for sphericity of the covariance matrix when the dimensionality, p, exceeds that of the sample size, N=n+1. Under the assumptions that (A) as p→∞ for i=1,…,16 and (B) p/n→c<∞ known as the concentration, a new statistic is developed utilizing the ratio of the fourth and second arithmetic means of the eigenvalues of the sample covariance matrix. The newly defined test has many desirable general asymptotic properties, such as normality and consistency when (n,p)→∞. Our simulation results show that the new test is comparable to, and in some cases more powerful than, the tests for sphericity in the current literature.