Estimation of normal covariance matrices parametrized by irreducible symmetric cones under Stein's loss

In this paper the problem of estimating a covariance matrix parametrized by an irreducible symmetric cone in a decision-theoretic set-up is considered. By making use of some results developed in a theory of finite-dimensional Euclidean simple Jordan algebras, Bartlett's decomposition and an unbiased risk estimate formula for a general family of Wishart distributions on the irreducible symmetric cone are derived; these results lead to an extension of Stein's general technique for derivation of minimax estimators for a real normal covariance matrix. Specification of the results to the multivariate normal models with covariances which are parametrized by complex, quaternion, and Lorentz types gives minimax estimators for each model.     

Estimation of the precision matrix of a singular Wishart distribution and its application in high-dimensional data

In this article, the Stein-Haff identity is established for a singular Wishart distribution with a positive definite mean matrix but with the dimension larger than the degrees of freedom. This identity is then used to obtain estimators of the precision matrix improving on the estimator based on the Moore-Penrose inverse of the Wishart matrix under the Efron-Morris loss function and its variants. Ridge-type empirical Bayes estimators of the precision matrix are also given and their dominance properties over the usual one are shown using this identity. Finally, these precision estimators are used in a quadratic discriminant rule, and it is shown through simulation that discriminant methods based on the ridge-type empirical Bayes estimators provide higher correct classification rates.     

Estimation of Wishart mean matrices under simple tree ordering

For Wishart density functions, we study the risk dominance problems of the restricted maximum likelihood estimators of mean matrices with respect to the Kullback-Leibler loss function over restricted parameter space under the simple tree ordering set. The results are directly applied to the estimation of covariance matrices for the completely balanced multivariate multi-way random effects models without interactions.     

On Bayes estimators with uniform priors on spheres and their comparative performance with maximum likelihood estimators for estimating bounded multivariate normal means

For independently distributed observables: X_i∼N(θ_i,σ²),i=1,…,p, we consider estimating the vector θ=(θ₁,…,θ_p)^′ with loss ‖d−θ‖² under the constraint , with known τ₁,…,τ_p,σ²,m. In comparing the risk performance of Bayesian estimators δ_α associated with uniform priors on spheres of radius α centered at (τ₁,…,τ_p) with that of the maximum likelihood estimator , we make use of Stein’s unbiased estimate of risk technique, Karlin’s sign change arguments, and a conditional risk analysis to obtain for a fixed (m,p) necessary and sufficient conditions on α for δ_α to dominate . Large sample determinations of these conditions are provided. Both cases where all such δ_α’s and cases where no such δ_α’s dominate are elicited. We establish, as a particular case, that the boundary uniform Bayes estimator δ_m dominates if and only if m≤k(p) with , improving on the previously known sufficient condition of Marchand and Perron (2001) [3] for which . Finally, we improve upon a universal dominance condition due to Marchand and Perron, by establishing that all Bayesian estimators δ_π with π spherically symmetric and supported on the parameter space dominate whenever m≤c₁(p) with .     

Estimation, prediction and the Stein phenomenon under divergence loss

We consider two problems: (1) estimate a normal mean under a general divergence loss introduced in [S. Amari, Differential geometry of curved exponential families — curvatures and information loss, Ann. Statist. 10 (1982) 357-387] and [N. Cressie, T.R.C. Read, Multinomial goodness-of-fit tests, J. Roy. Statist. Soc. Ser. B. 46 (1984) 440-464] and (2) find a predictive density of a new observation drawn independently of observations sampled from a normal distribution with the same mean but possibly with a different variance under the same loss. The general divergence loss includes as special cases both the Kullback-Leibler and Bhattacharyya-Hellinger losses. The sample mean, which is a Bayes estimator of the population mean under this loss and the improper uniform prior, is shown to be minimax in any arbitrary dimension. A counterpart of this result for predictive density is also proved in any arbitrary dimension. The admissibility of these rules holds in one dimension, and we conjecture that the result is true in two dimensions as well. However, the general Baranchick [A.J. Baranchick, a family of minimax estimators of the mean of a multivariate normal distribution, Ann. Math. Statist. 41 (1970) 642-645] class of estimators, which includes the James-Stein estimator and the Strawderman [W.E. Strawderman, Proper Bayes minimax estimators of the multivariate normal mean, Ann. Math. Statist. 42 (1971) 385-388] class of estimators, dominates the sample mean in three or higher dimensions for the estimation problem. An analogous class of predictive densities is defined and any member of this class is shown to dominate the predictive density corresponding to a uniform prior in three or higher dimensions. For the prediction problem, in the special case of Kullback-Leibler loss, our results complement to a certain extent some of the recent important work of Komaki [F. Komaki, A shrinkage predictive distribution for multivariate normal observations, Biometrika 88 (2001) 859-864] and George, Liang and Xu [E.I. George, F. Liang, X. Xu, Improved minimax predictive densities under Kullbak-Leibler loss, Ann. Statist. 34 (2006) 78-92]. While our proposed approach produces a general class of predictive densities (not necessarily Bayes, but not excluding Bayes predictors) dominating the predictive density under a uniform prior. We show also that various modifications of the James-Stein estimator continue to dominate the sample mean, and by the duality of estimation and predictive density results which we will show, similar results continue to hold for the prediction problem as well.     

On Stein’s lemma, dependent covariates and functional monotonicity in multi-dimensional modeling

Tracking the correct directions of monotonicity in multi-dimensional modeling plays an important role in interpreting functional associations. In the presence of multiple predictors, we provide empirical evidence that the observed monotone directions via parametric, nonparametric or semiparametric fit of commonly used multi-dimensional models may entirely violate the actual directions of monotonicity. This breakdown is caused primarily by the dependence structure of covariates, with negligible influence from the bias of function estimation. To examine the linkage between the dependent covariates and monotone directions, we first generalize Stein’s Lemma for random variables which are mutually independent Gaussian to two important cases: dependent Gaussian, and independent non-Gaussian. We show that in both two cases, there is an explicit one-to-one correspondence between the monotone directions of a multi-dimensional function and the signs of a deterministic surrogate vector. Moreover, we demonstrate that the second case can be extended to accommodate a class of dependent covariates. This generalization further enables us to develop a de-correlation transform for arbitrarily dependent covariates. The transformed covariates preserve modeling interpretability with little loss in modeling efficiency. The simplicity and effectiveness of the proposed method are illustrated via simulation studies and real data application.     

Bootstrap of deviation probabilities with applications

We show that under different moment bounds on the underlying variables, bootstrap approximation to the large deviation probabilities of standardized sample sum, based on independent random variables, is valid for a wider zone of n, the sample size, compared to the classical normal tail probability approximation. As an application, different notions of efficiency for statistical tests are considered from Bayesian point of view. In particular, efficiency due to Pitman (1938) [11], Chernoff (1952) [1], and Bayes risk efficiency due to Rubin and Sethuraman (1965) [12] turn out to be special cases with the choice of the weight function; i.e., prior density times loss.     

Estimation of means of multivariate normal populations with order restriction

Multivariate isotonic regression theory plays a key role in the field of statistical inference under order restriction for vector valued parameters. Two cases of estimating multivariate normal means under order restricted set are considered. One case is that covariance matrices are known, the other one is that covariance matrices are unknown but are restricted by partial order. This paper shows that when covariance matrices are known, the estimator given by this paper always dominates unrestricted maximum likelihood estimator uniformly, and when covariance matrices are unknown, the plug-in estimator dominates unrestricted maximum likelihood estimator under the order restricted set of covariance matrices. The isotonic regression estimators in this paper are the generalizations of plug-in estimators in unitary case.     

Generalized Bayes minimax estimators of the mean of multivariate normal distribution with unknown variance

We construct a broad class of generalized Bayes minimax estimators of the mean of a multivariate normal distribution with covariance equal to σ²I_p, with σ² unknown, and under the invariant loss δ(X)−θ²/σ². Examples that illustrate the theory are given. Most notably it is shown that a hierarchical version of the multivariate Student-t prior yields a Bayes minimax estimate.     

Existence conditions for the uniformly minimum risk unbiased estimators in a class of linear models

This paper studies the existence of the uniformly minimum risk unbiased (UMRU) estimators of parameters in a class of linear models with an error vector having multivariate normal distribution or t-distribution, which include the growth curve model, the extended growth curve model, the seemingly unrelated regression equations model, the variance components model, and so on. The necessary and sufficient existence conditions are established for UMRU estimators of the estimable linear functions of regression coefficients under convex losses and matrix losses, respectively. Under the (extended) growth curve model and the seemingly unrelated regression equations model with normality assumption, the conclusions given in the literature can be derived by applying the general results in this paper. For the variance components model, the necessary and sufficient existence conditions are reduced as terse forms.     

Finite sample tail behavior of multivariate location estimators

A finite sample performance measure of multivariate location estimators is introduced based on “tail behavior”. The tail performance of multivariate “monotone” location estimators and the halfspace depth based “non-monotone” location estimators including the Tukey halfspace median and multivariate L-estimators is investigated. The connections among the finite sample performance measure, the finite sample breakdown point, and the halfspace depth are revealed. It turns out that estimators with high breakdown point or halfspace depth have “appealing” tail performance. The tail performance of the halfspace median is very appealing and also robust against underlying population distributions, while the tail performance of the sample mean is very sensitive to underlying population distributions. These findings provide new insights into the notions of the halfspace depth and breakdown point and identify the important role of tail behavior as a quantitative measure of robustness in the multivariate location setting.     

Robust discrimination under a hierarchy on the scatter matrices

Under normality, Flury and Schmid [Quadratic discriminant functions with constraints on the covariances matrices: some asymptotic results, J. Multivariate Anal. 40 (1992) 244-261] investigated the asymptotic properties of the quadratic discrimination procedure under hierarchical models for the scatter matrices, that is: (i) arbitrary scatter matrices, (ii) common principal components, (iii) proportional scatter matrices and (iv) identical matrices. In this paper, we study the properties of robust quadratic discrimination rules based on robust estimates of the involved parameters. Our analysis is based on the partial influence functions of the functionals related to these parameters and allows to derive the asymptotic variances of the estimated coefficients under models (i)-(iv). From them, we conclude that the asymptotic variances verify the same order relations as those obtained by Flury and Schmid [Quadratic discriminant functions with constraints on the covariances matrices: some asymptotic results, J. Multivariate Anal. 40 (1992) 244-261] for the classical estimators. We also perform a Monte Carlo study for different sample sizes and different hierarchies which shows the advantage of using robust procedures over classical ones, when anomalous data are present. It also confirms that better rates of misclassification can be achieved if a more parsimonious model among all the correct ones is used instead of the standard quadratic discrimination.     

Admissibility and minimaxity of generalized Bayes estimators for spherically symmetric family

We give a sufficient condition for admissibility of generalized Bayes estimators of the location vector of spherically symmetric distribution under squared error loss. Compared to the known results for the multivariate normal case, our sufficient condition is very tight and is close to being a necessary condition. In particular, we establish the admissibility of generalized Bayes estimators with respect to the harmonic prior and priors with slightly heavier tail than the harmonic prior. We use the theory of regularly varying functions to construct a sequence of smooth proper priors approaching an improper prior fast enough for establishing the admissibility. We also discuss conditions of minimaxity of the generalized Bayes estimator with respect to the harmonic prior.     

Admissibility and minimaxity of Bayes estimators for a normal mean matrix

In some invariant estimation problems under a group, the Bayes estimator against an invariant prior has equivariance as well. This is useful notably for evaluating the frequentist risk of the Bayes estimator. This paper addresses the problem of estimating a matrix of means in normal distributions relative to quadratic loss. It is shown that a matricial shrinkage Bayes estimator against an orthogonally invariant hierarchical prior is admissible and minimax by means of equivariance. The analytical improvement upon every over-shrinkage equivariant estimator is also considered and this paper justifies the corresponding positive-part estimator preserving the order of the sample singular values.     

Proper bayes minimax estimators for a multivariate normal mean with unknown common variance under a convex loss function

Summary Let X ∼ N_p(μ,σ²I_p) and let s/σ² ∼ χ _n ² , independent ofX, where μ and σ² are unknown. This paper considers the estimation of μ (by δ) relative to a convex loss function given by (δ−μ)′[(1−α)I_p/σ²+αQ](δ−μ)/[(1−α)p/σ²+α tr (Q)], whereQ is a knownp×p diagonal matrix and 0≦α≦1. Two classes of minimax estimators are obtained for μ whenp≦3; the first is a new result and the second is a generalization of a result of Strawderman (1973,Ann. Statist.,1, 1189–1194). A proper Bayes estimator is also obtained which is shown to satisfy the conditions of the second class of minimax estimators. The paper concludes by discussing the estimation of μ relative to another convex loss function. This work was supported by the Army, Navy and Air Force under Office of Naval Research Contract No. N00014-80-C-0093. Reproduction in whole or in part is permitted for any purpose of the United States Government.     

The Stein phenomenon for monotone incomplete multivariate normal data

We establish the Stein phenomenon in the context of two-step, monotone incomplete data drawn from , a (p+q)-dimensional multivariate normal population with mean and covariance matrix . On the basis of data consisting of n observations on all p+q characteristics and an additional N−n observations on the last q characteristics, where all observations are mutually independent, denote by the maximum likelihood estimator of . We establish criteria which imply that shrinkage estimators of James-Stein type have lower risk than under Euclidean quadratic loss. Further, we show that the corresponding positive-part estimators have lower risk than their unrestricted counterparts, thereby rendering the latter estimators inadmissible. We derive results for the case in which is block-diagonal, the loss function is quadratic and non-spherical, and the shrinkage estimator is constructed by means of a nondecreasing, differentiable function of a quadratic form in . For the problem of shrinking to a vector whose components have a common value constructed from the data, we derive improved shrinkage estimators and again determine conditions under which the positive-part analogs have lower risk than their unrestricted counterparts.     

Improving on the maximum likelihood estimators of the means in Poisson decomposable graphical models

In this article we study the simultaneous estimation of the means in Poisson decomposable graphical models. We derive some classes of estimators which improve on the maximum likelihood estimator under the normalized squared losses. Our estimators are based on the argument in Chou [Simultaneous estimation in discrete multivariate exponential families, Ann. Statist. 19 (1991) 314-328.] and shrink the maximum likelihood estimator depending on the marginal frequencies of variables forming a complete subgraph of the conditional independence graph.     

Generalized Bayes minimax estimators of location vectors for spherically symmetric distributions

Let X∼f(∥x-θ∥²) and let δ_π(X) be the generalized Bayes estimator of θ with respect to a spherically symmetric prior, π(∥θ∥²), for loss ∥δ-θ∥². We show that if π(t) is superharmonic, non-increasing, and has a non-decreasing Laplacian, then the generalized Bayes estimator is minimax and dominates the usual minimax estimator δ₀(X)=X under certain conditions on . The class of priors includes priors of the form for and hence includes the fundamental harmonic prior . The class of sampling distributions includes certain variance mixtures of normals and other functions f(t) of the form e^-αtβ and e^-αt+βφ(t) which are not mixtures of normals. The proofs do not rely on boundness or monotonicity of the function r(t) in the representation of the Bayes estimator as .