Minimax and admissible minimax estimators of the mean of a multivariate normal distribution for unknown covariance matrix

Let X be a p-variate (p ≥ 3) vector normally distributed with mean μ and covariance Σ, and let A be a p × p random matrix distributed independent of X, according to the Wishart distribution W(n, Σ). For estimating μ, we consider estimators of the form δ = δ(X, A). We obtain families of Bayes, minimax and admissible minimax estimators with respect to the quadratic loss function (δ ? μ)′ Σ^?1(δ ? μ) where Σ is unknown. This paper extends previous results of the author [1], given for the case in which the covariance matrix of the distribution is of the form σ²I, where σ is known.     

Estimating a Cholesky decomposition

For a normal distribution the sample covariance matrix S provides an unbiased estimator of the population covariance matrix Σ. We address the problem of finding an unbiased estimator of the lower triangular matrix Ψ defined by the Cholesky decomposition Σ = ΨΨ′.     

Some tests for the covariance matrix with fewer observations than the dimension under non-normality

This article analyzes whether some existing tests for the p×p covariance matrix Σ of the N independent identically distributed observation vectors work under non-normality. We focus on three hypotheses testing problems: (1) testing for sphericity, that is, the covariance matrix Σ is proportional to an identity matrix I_p; (2) the covariance matrix Σ is an identity matrix I_p; and (3) the covariance matrix is a diagonal matrix. It is shown that the tests proposed by Srivastava (2005) for the above three problems are robust under the non-normality assumption made in this article irrespective of whether N≤p or N≥p, but (N,p)→∞, and N/p may go to zero or infinity. Results are asymptotic and it may be noted that they may not hold for finite (N,p).     

Asymptotic distributions of functions of the eigenvalues of sample covariance matrix and canonical correlation matrix in multivariate time series

Let S = (1/n) Σ_t=1ⁿ X(t) X(t)′, where X(1), …, X(n) are p × 1 random vectors with mean zero. When X(t) (t = 1, …, n) are independently and identically distributed (i.i.d.) as multivariate normal with mean vector 0 and covariance matrix Σ, many authors have investigated the asymptotic expansions for the distributions of various functions of the eigenvalues of S. In this paper, we will extend the above results to the case when {X(t)} is a Gaussian stationary process. Also we shall derive the asymptotic expansions for certain functions of the sample canonical correlations in multivariate time series. Applications of some of the results in signal processing are also discussed.     

On the power of Wilks' U-test for MANOVA

Let V₁,…, V_m, W₁,…, W_n be independent p × 1 random vectors having multivariate normal distributions with common nonsingular covariance matrix Σ and with EW_α = 0, α = 1,…, n. In this canonical form of the multivariate linear model, the problem is to test H: EV_αazμ_α = 0, α = 1,…, m vs K: not H. It is shown that when the rank of the noncentrality matrix (μ₁…μ_m) Σ^?1 (μ₁…μ_m) is one, the power of Wilks' U-test (the likelihood ratio test) strictly decreases with the dimension p and the hypothesis degrees of freedom m. This generalizes results known for the noncentral F-test in the univariate case.     

Estimation of multivariate normal covariance and precision matrices in a star-shape model with missing data

In this paper, we study the problem of estimating the covariance matrix Σ and the precision matrix Ω (the inverse of the covariance matrix) in a star-shape model with missing data. By considering a type of Cholesky decomposition of the precision matrix Ω=Ψ^′Ψ, where Ψ is a lower triangular matrix with positive diagonal elements, we get the MLEs of the covariance matrix and precision matrix and prove that both of them are biased. Based on the MLEs, unbiased estimators of the covariance matrix and precision matrix are obtained. A special group G, which is a subgroup of the group consisting all lower triangular matrices, is introduced. By choosing the left invariant Haar measure on G as a prior, we obtain the closed forms of the best equivariant estimates of Ω under any of the Stein loss, the entropy loss, and the symmetric loss. Consequently, the MLE of the precision matrix (covariance matrix) is inadmissible under any of the above three loss functions. Some simulation results are given for illustration.     

Distribution of the likelihood ratio criterion for testing Σ = Σ0, μ = μ0

The exact null distribution of the likelihood ratio criterion for testing H₀: Σ = Σ₀ and μ = μ₀ against alternatives H₁: Σ ≠ Σ₀ or μ ≠ μ₀ in N_p(μ, Σ) has been obtained as (a) a chi-square series and (b) a beta series. Percentage points have been tabulated for p = 2(1) 6, α = .005, .01, .025, .05, .1, and .25 and various values of sample size N.     

Robustness of A-optimal designs

Suppose that Y=(Y_i) is a normal random vector with mean Xb and covariance σ²I_n, where b is a p-dimensional vector (b_j),X=(X_ij) is an n×p matrix. A-optimal designs X are chosen from the traditional set D of A-optimal designs for ρ=0 such that X is still A-optimal in D when the components Y_i are dependent, i.e., for i≠i′, the covariance of Y_i,Y_i^′ is ρ with ρ≠0. Such designs depend on the sign of ρ. The general results are applied to X=(X_ij), where X_ij∈{-1,1}; this corresponds to a factorial design with -1,1 representing low level or high level respectively, or corresponds to a weighing design with -1,1 representing an object j with weight b_j being weighed on the left and right of a chemical balance respectively.     

Contracting towards subspaces when estimating the mean of a multivariate normal distribution

The problem of estimating, under unweighted quadratic loss, the mean of a multinormal random vector X with arbitrary covariance matrix V is considered. The results of James and Stein for the case V = I have since been extended by Bock to cover arbitrary V and also to allow for contracting X towards a subspace other than the origin; minimax estimators (other than X) exist if and only if the eigenvalues of V are not “too spread out.” In this paper a slight variation of Bock's estimator is considered. A necessary and sufficient condition for the minimaxity of the present estimator is (1): the eigenvalues of (I ? P) V should not be “too spread out,” where P denotes the projection matrix associated with the subspace towards which X is contracted. The validity of (1) is then examined for a number of patterned covariance matrices (e.g., intraclass covariance, tridiagonal and first order autocovariance) and conditions are given for (1) to hold when contraction is towards the origin or towards the common mean of the components of X. (1) is also examined when X is the usual estimate of the regression vector in multiple linear regression. In several of the cases considered the eigenvalues of V are “too spread out” while those of (I ? P) V are not, so that in these instances the present method can be used to produce a minimax estimate.     

Wishart ratios with dependent structure: New members of the bimatrix beta type IV

In multivariate statistics under normality, the problems of interest are random covariance matrices (known as Wishart matrices) and “ratios” of Wishart matrices that arise in multivariate analysis of variance (MANOVA) (see 24). The bimatrix variate beta type IV distribution (also known in the literature as bimatrix variate generalised beta; matrix variate generalization of a bivariate beta type I) arises from “ratios” of Wishart matrices. In this paper, we add a further independent Wishart random variate to the “denominator” of one of the ratios; this results in deriving the exact expression for the density function of the bimatrix variate extended beta type IV distribution. The latter leads to the proposal of the bimatrix variate extended F distribution. Some interesting characteristics of these newly introduced bimatrix distributions are explored. Lastly, we focus on the bivariate extended beta type IV distribution (that is an extension of bivariate Jones’ beta) with emphasis on P(X₁<X₂) where X₁ is the random stress variate and X₂ is the random strength variate.     

Non-white Wishart ensembles

We consider non-white Wishart ensembles , where X is a p×N random matrix with i.i.d. complex standard Gaussian entries and Σ is a covariance matrix, with fixed eigenvalues, close to the identity matrix. We prove that the largest eigenvalue of such random matrix ensembles exhibits a universal behavior in the large-N limit, provided Σ is “close enough” to the identity matrix. If not, we identify the limiting distribution of the largest eigenvalues, focusing on the case where the largest eigenvalues almost surely exit the support of the limiting Marchenko-Pastur's distribution.     

Unbiased invariant minimum norm estimation in generalized growth curve model

This paper considers the generalized growth curve model subject to R(X_m)⊆R(X_m-1)⊆?⊆R(X₁), where B_i are the matrices of unknown regression coefficients, X_i,Z_i and U are known covariate matrices, i=1,2,…,m, and E splits into a number of independently and identically distributed subvectors with mean zero and unknown covariance matrix Σ. An unbiased invariant minimum norm quadratic estimator (MINQE(U,I)) of tr(CΣ) is derived and the conditions for its optimality under the minimum variance criterion are investigated. The necessary and sufficient conditions for MINQE(U,I) of tr(CΣ) to be a uniformly minimum variance invariant quadratic unbiased estimator (UMVIQUE) are obtained. An unbiased invariant minimum norm quadratic plus linear estimator (MINQLE(U,I)) of is also given. To compare with the existing maximum likelihood estimator (MLE) of tr(CΣ), we conduct some simulation studies which show that our proposed estimator performs very well.     

Multivariate model with a Kronecker product covariance structure: S. N. Roy method of estimation

In this paper we consider a (p × q)-matrix X = (X ₁, ..., X _q ), where a pq-vector vec (X) = (X ₁ ^T , ...,X _q ^T ) ^T is assumed to be distributed normally with mean vector vec (M) = (M ₁ ^T , ...,M _q ^T ) ^T and a positive definite covariance matrix Λ. Suppose that Λ follows a Kronecker product covariance structure, that is Λ = Φ?Σ, where Φ = (? _ij ) is a (q × q)-matrix and Σ = (σ _ij ) is a (p × p)-matrix and the matrices Φ, Σ are positive definite. Such a model is considered in [4], where the maximum likelihood estimates of the parameters M, Φ, Σ are obtained. Using S. N. Roy’s technique (see, e.g., [3]) of the multivariate statistical analysis, we obtain consistent and unbiased estimates of M, Φ, Σ as in [4], but with less calculations.     

The distance between two random vectors with given dispersion matrices

For two p-dimensional random vectors X and Y with dispersion matrices Σ₁₁ and Σ₂₂, respectively, we determine that covariance matrix Ψ₀ of X and Y that minimizes the L₂-distance between X and Y. There is a dual to this problem that is of interest in another context.     

On testing for clusters using the sample covariance

This paper analyzes the problem of using the sample covariance matrix to detect the presence of clustering in p-variate data in the special case when the component covariance matrices are known up to a constant multiplier. For the case of testing one population against a mixture of two populations, tests are derived and shown to be optimal in a certain sense. Some of their distribution properties are derived exactly. Some remarks on the extensions of these tests to mixtures of k ≤ p populations are included. The paper is essentially a formal treatment (in a special case) of some well-known procedures. The methods used in deriving the distribution properties are applicable to a variety of other situations involving mixtures.     

A characterization of the multivariate normal distribution

Let X_j (j = 1,…,n) be i.i.d. random variables, and let Y′ = (Y₁,…,Y_m) and X′ = (X₁,…,X_n) be independently distributed, and A = (a_jk) be an n × n random coefficient matrix with a_jk = a_jk(Y) for j, k = 1,…,n. Consider the equation U = AX, Kingman and Graybill [Ann. Math. Statist.41 (1970)] have shown U ～ N(O,I) if and only if X ～ N(O,I). provided that certain conditions defined in terms of the a_jk are satisfied. The task of this paper is to delete the identical assumption on X₁,…,X_n and then generalize the results to the vector case. Furthermore, the condition of independence on the random components within each vector is relaxed, and also the question raised by the above authors is answered.     

Representations of best linear unbiased estimators in the Gauss-Markoff model with a singular dispersion matrix

In the general Gauss-Markoff model (Y, Xβ, σ²V), when V is singular, there exist linear functions of Y which vanish with probability 1 imposing some restrictions on Y as well as on the unknown β. In all earlier work on linear estimation, representations of best-linear unbiased estimators (BLUE's) are obtained under the assumption: “L′Y is unbiased for Xβ ? L′X = X.” Such a condition is not, however, necessary. The present paper provides all possible representations of the BLUE's some of which violate the condition L′X = X. Representations of X for given classes of BLUE's are also obtained.     

Minimax estimation of a multivariate normal mean under polynomial loss

Let X be an observation from a p-variate (p ≥ 3) normal random vector with unknown mean vector θ and known covariance matrix

. The problem of improving upon the usual estimator of θ, δ⁰(X) = X, is considered. An approach is developed which can lead to improved estimators, δ, for loss functions which are polynomials in the coordinates of (δ ? θ). As an example of this approach, the loss L(δ, θ) = |δ ? θ|⁴ is considered, and estimators are developed which are significantly better than δ⁰. When

is the identity matrix, these estimators are of the form $\text{δ(X) = (1 ? (}\text{b}\text{(d + |X|}{}^2\text{)}\text{))X}$.     

Multivariate analysis of variance with fewer observations than the dimension

In this article, we consider the problem of testing a linear hypothesis in a multivariate linear regression model which includes the case of testing the equality of mean vectors of several multivariate normal populations with common covariance matrix Σ, the so-called multivariate analysis of variance or MANOVA problem. However, we have fewer observations than the dimension of the random vectors. Two tests are proposed and their asymptotic distributions under the hypothesis as well as under the alternatives are given under some mild conditions. A theoretical comparison of these powers is made.     

Trimmed estimates in simultaneous estimation of parameters in exponential families

Let X₁,…, X_p be p (≥ 3) independent random variables, where each X_i has a distribution belonging to the one-parameter exponential family of distributions. The problem is to estimate the unknown parameters simultaneously in the presence of extreme observations. C. Stein (Ann. Statist.9 (1981), 1135–1151) proposed a method of estimating the mean vector of a multinormal distribution, based on order statistics corresponding to the |X_i|'s, which permitted improvement over the usual maximum likelihood estimator, for long-tailed empirical distribution functions. In this paper, the ideas of Stein are extended to the general discrete and absolutely continuous exponential families of distributions. Adaptive versions of the estimators are also discussed.