Estimation of the precision matrix of a singular Wishart distribution and its application in high-dimensional data

In this article, the Stein-Haff identity is established for a singular Wishart distribution with a positive definite mean matrix but with the dimension larger than the degrees of freedom. This identity is then used to obtain estimators of the precision matrix improving on the estimator based on the Moore-Penrose inverse of the Wishart matrix under the Efron-Morris loss function and its variants. Ridge-type empirical Bayes estimators of the precision matrix are also given and their dominance properties over the usual one are shown using this identity. Finally, these precision estimators are used in a quadratic discriminant rule, and it is shown through simulation that discriminant methods based on the ridge-type empirical Bayes estimators provide higher correct classification rates.     

Estimating the covariance matrix: a new approach

In this paper, we consider the problem of estimating the covariance matrix and the generalized variance when the observations follow a nonsingular multivariate normal distribution with unknown mean. A new method is presented to obtain a truncated estimator that utilizes the information available in the sample mean matrix and dominates the James-Stein minimax estimator. Several scale equivariant minimax estimators are also given. This method is then applied to obtain new truncated and improved estimators of the generalized variance; it also provides a new proof to the results of Shorrock and Zidek (Ann. Statist. 4 (1976) 629) and Sinha (J. Multivariate Anal. 6 (1976) 617).     

Modifying estimators of ordered positive parameters under the Stein loss

This paper treats the problem of estimating positive parameters restricted to a polyhedral convex cone which includes typical order restrictions, such as simple order, tree order and umbrella order restrictions. In this paper, two methods are used to show the improvement of order-preserving estimators over crude non-order-preserving estimators without any assumption on underlying distributions. One is to use Fenchel’s duality theorem, and then the superiority of the isotonic regression estimator is established under the general restriction to polyhedral convex cones. The use of the Abel identity is the other method, and we can derive a class of improved estimators which includes order-statistics-based estimators in the typical order restrictions. When the underlying distributions are scale families, the unbiased estimators and their order-restricted estimators are shown to be minimax. The minimaxity of the restrictedly generalized Bayes estimator against the prior over the restricted space is also demonstrated in the two dimensional case. Finally, some examples and multivariate extensions are given.     

Admissibility and minimaxity of Bayes estimators for a normal mean matrix

In some invariant estimation problems under a group, the Bayes estimator against an invariant prior has equivariance as well. This is useful notably for evaluating the frequentist risk of the Bayes estimator. This paper addresses the problem of estimating a matrix of means in normal distributions relative to quadratic loss. It is shown that a matricial shrinkage Bayes estimator against an orthogonally invariant hierarchical prior is admissible and minimax by means of equivariance. The analytical improvement upon every over-shrinkage equivariant estimator is also considered and this paper justifies the corresponding positive-part estimator preserving the order of the sample singular values.     

A comparison of minimax and least squares estimators in linear regression with polyhedral prior information

We consider the linear regression model where prior information in the form of linear inequalities restricts the parameter space to a polyhedron. Since the linear minimax estimator has, in general, to be determined numerically, it was proposed to minimize an upper bound of the maximum risk instead. The resulting so-called quasiminimax estimator can be easily calculated in closed form. Unfortunately, both minimax estimators may violate the prior information. Therefore, we consider projection estimators which are obtained by projecting the estimate in an optional second step. The performance of these estimators is investigated in a Monte Carlo study together with several least squares estimators, including the inequality restricted least squares estimator. It turns out that both the projected and the unprojected quasiminimax estimators have the best average performance.     

A subclass of bayes linear estimators that are minimax

In the linear regression model with ellipsoidal parameter constraints, the problem of estimating the unknown parameter vector is studied. A well-described subclass of Bayes linear estimators is proposed in the paper. It is shown that for each member of this subclass, a generalized quadratic risk function exists so that the estimator is minimax. Moreover, some of the proposed Bayes linear estimators are admissible with respect to all possible generalized quadratic risks. Also, a necessary and sufficient condition is given to ensure that the considered Bayes linear estimator improves the least squares estimator over the whole ellipsoid whatever generalized risk function is chosen.     

On improved estimation of normal precision matrix and discriminant coefficients

The problem of estimating the precision matrix of a multivariate normal distribution model is considered with respect to a quadratic loss function. A number of covariance estimators originally intended for a variety of loss functions are adapted so as to obtain alternative estimators of the precision matrix. It is shown that the alternative estimators have analytically smaller risks than the unbiased estimator of the precision matrix. Through numerical studies of risk values, it is shown that the new estimators have substantial reduction in risk. In addition, we consider the problem of the estimation of discriminant coefficients, which arises in linear discriminant analysis when Fisher's linear discriminant function is viewed as the posterior log-odds under the assumption that two classes differ in mean but have a common covariance matrix. The above method is also adapted for this problem in order to obtain improved estimators of the discriminant coefficients under the quadratic loss function. Furthermore, a numerical study is undertaken to compare the properties of a collection of alternatives to the “unbiased” estimator of the discriminant coefficients.     

Shrinkage priors for Bayesian estimation of the mean matrix in an elliptically contoured distribution

This paper deals with the problem of estimating the mean matrix in an elliptically contoured distribution with unknown scale matrix. The Laplace and inverse Laplace transforms of the density allow us not only to evaluate the risk function with respect to a quadratic loss but also to simplify expressions of Bayes estimators. Consequently, it is shown that generalized Bayes estimators against shrinkage priors dominate the unbiased estimator.     

Estimation of normal covariance matrices parametrized by irreducible symmetric cones under Stein's loss

In this paper the problem of estimating a covariance matrix parametrized by an irreducible symmetric cone in a decision-theoretic set-up is considered. By making use of some results developed in a theory of finite-dimensional Euclidean simple Jordan algebras, Bartlett's decomposition and an unbiased risk estimate formula for a general family of Wishart distributions on the irreducible symmetric cone are derived; these results lead to an extension of Stein's general technique for derivation of minimax estimators for a real normal covariance matrix. Specification of the results to the multivariate normal models with covariances which are parametrized by complex, quaternion, and Lorentz types gives minimax estimators for each model.     

Stein's phenomenon in estimation of means restricted to a polyhedral convex cone

This paper treats the problem of estimating the restricted means of normal distributions with a known variance, where the means are restricted to a polyhedral convex cone which includes various restrictions such as positive orthant, simple order, tree order and umbrella order restrictions. In the context of the simultaneous estimation of the restricted means, it is of great interest to investigate decision-theoretic properties of the generalized Bayes estimator against the uniform prior distribution over the polyhedral convex cone. In this paper, the generalized Bayes estimator is shown to be minimax. It is also proved that it is admissible in the one- or two-dimensional case, but is improved on by a shrinkage estimator in the three- or more-dimensional case. This means that the so-called Stein phenomenon on the minimax generalized Bayes estimator can be extended to the case where the means are restricted to the polyhedral convex cone. The risk behaviors of the estimators are investigated through Monte Carlo simulation, and it is revealed that the shrinkage estimator has a substantial risk reduction.     

Improved estimation of a covariance matrix in an elliptically contoured matrix distribution

In this paper, the problem of estimating the covariance matrix of the elliptically contoured distribution (ECD) is considered. A new class of estimators which shrink the eigenvalues towards their arithmetic mean is proposed. It is shown that this new estimator dominates the unbiased estimator under the squared error loss function. Two special classes of ECD, namely, the multivariate-elliptical t distribution and the ε-contaminated normal distribution are considered. A simulation study is carried out and indicates that this new shrinkage estimator provides a substantial improvement in risk under most situations.     

Sharp adaptive estimation of quadratic functionals

Estimation of a quadratic functional of a function observed in the Gaussian white noise model is considered. A data-dependent method for choosing the amount of smoothing is given. The method is based on comparing certain quadratic estimators with each other. It is shown that the method is asymptotically sharp or nearly sharp adaptive simultaneously for the “regular” and “irregular” region. We consider l_p bodies and construct bounds for the risk of the estimator which show that for p=4 the estimator is exactly optimal and for example when p ∈[3,100], then the upper bound is at most 1.055 times larger than the lower bound. We show the connection of the estimator to the theory of optimal recovery. The estimator is a calibration of an estimator which is nearly minimax optimal among quadratic estimators. Writing of this article was financed by Deutsche Forschungsgemeinschaft under project MA1026/6-2, CIES, France, and Jenny and AnttiWihuri Foundation.     

Estimation, prediction and the Stein phenomenon under divergence loss

We consider two problems: (1) estimate a normal mean under a general divergence loss introduced in [S. Amari, Differential geometry of curved exponential families — curvatures and information loss, Ann. Statist. 10 (1982) 357-387] and [N. Cressie, T.R.C. Read, Multinomial goodness-of-fit tests, J. Roy. Statist. Soc. Ser. B. 46 (1984) 440-464] and (2) find a predictive density of a new observation drawn independently of observations sampled from a normal distribution with the same mean but possibly with a different variance under the same loss. The general divergence loss includes as special cases both the Kullback-Leibler and Bhattacharyya-Hellinger losses. The sample mean, which is a Bayes estimator of the population mean under this loss and the improper uniform prior, is shown to be minimax in any arbitrary dimension. A counterpart of this result for predictive density is also proved in any arbitrary dimension. The admissibility of these rules holds in one dimension, and we conjecture that the result is true in two dimensions as well. However, the general Baranchick [A.J. Baranchick, a family of minimax estimators of the mean of a multivariate normal distribution, Ann. Math. Statist. 41 (1970) 642-645] class of estimators, which includes the James-Stein estimator and the Strawderman [W.E. Strawderman, Proper Bayes minimax estimators of the multivariate normal mean, Ann. Math. Statist. 42 (1971) 385-388] class of estimators, dominates the sample mean in three or higher dimensions for the estimation problem. An analogous class of predictive densities is defined and any member of this class is shown to dominate the predictive density corresponding to a uniform prior in three or higher dimensions. For the prediction problem, in the special case of Kullback-Leibler loss, our results complement to a certain extent some of the recent important work of Komaki [F. Komaki, A shrinkage predictive distribution for multivariate normal observations, Biometrika 88 (2001) 859-864] and George, Liang and Xu [E.I. George, F. Liang, X. Xu, Improved minimax predictive densities under Kullbak-Leibler loss, Ann. Statist. 34 (2006) 78-92]. While our proposed approach produces a general class of predictive densities (not necessarily Bayes, but not excluding Bayes predictors) dominating the predictive density under a uniform prior. We show also that various modifications of the James-Stein estimator continue to dominate the sample mean, and by the duality of estimation and predictive density results which we will show, similar results continue to hold for the prediction problem as well.     

Statistical inference in a panel data semiparametric regression model with serially correlated errors

We consider a panel data semiparametric partially linear regression model with an unknown vector β of regression coefficients, an unknown nonparametric function g(·) for nonlinear component, and unobservable serially correlated errors. The correlated errors are modeled by a vector autoregressive process which involves a constant intraclass correlation. Applying the pilot estimators of β and g(·), we construct estimators of the autoregressive coefficients, the intraclass correlation and the error variance, and investigate their asymptotic properties. Fitting the error structure results in a new semiparametric two-step estimator of β, which is shown to be asymptotically more efficient than the usual semiparametric least squares estimator in terms of asymptotic covariance matrix. Asymptotic normality of this new estimator is established, and a consistent estimator of its asymptotic covariance matrix is presented. Furthermore, a corresponding estimator of g(·) is also provided. These results can be used to make asymptotically efficient statistical inference. Some simulation studies are conducted to illustrate the finite sample performances of these proposed estimators.     

Linear minimax estimation with ellipsoidal constraints

We consider the simultaneous linear minimax estimation problem in linear models with ellipsoidal constraints imposed on an unknown parameter. Using convex analysis, we derive necessary and sufficient optimality conditions for a matrix to define the linear minimax estimator. For certain regions of the set of characteristics of linear models and constraints, we exploit these optimality conditions and get explicit formulae for linear minimax estimators.     

Multivariate Failure Times Regression with a Continuous Auxiliary Covariate

How to take advantage of the available auxiliary covariate information when the primary covariate of interest is not measured is a frequently encountered question in biomedical study. In this paper, we consider the multivariate failure times regression analysis in which the primary covariate is assessed only in a validation set, but a continuous auxiliary covariate for it is available for all subjects in the study cohort. Under the frame of marginal hazard model, we propose to estimate the induced relative risk function in the non-validation set through kernel smoothing method and then obtain an estimated pseudo-partial likelihood function. The proposed estimator which maximizes the estimated pseudo-partial likelihood is shown to be consistent and asymptotically normal. We also give an estimator of the marginal cumulative baseline hazard function. Simulation studies are conducted to evaluate the finite sample performance of our proposed estimator. The proposed method is illustrated by analyzing a heart disease data from the Study of Left Ventricular Dysfunction (SOLVD).     

Simultaneous estimation of Poisson means

In the simultaneous estimation of means from independent Poisson distributions, an estimator is developed which incorporates a prior mean and variance for each Poisson mean estimated. This estimator possesses substantially smaller risk than the usual estimator in a region of the parameter space and seems superior to other estimators proposed to estimate p Poisson means. It is indicated through two asymptotic results that, unlike the conjugate Bayes estimator, the risk of the estimator does not greatly exceed the risk of the usual estimator outside of the region of risk improvement.     

Orthogonally invariant estimation of the skew-symmetric normal mean matrix

The unbiased estimator of risk of the orthogonally invariant estimator of the skew-symmetric normal mean matrix is obtained, and a class of minimax estimators and their order-preserving modification are proposed. The estimators have applications in paired comparisons model. A Monte Carlo study to compare the risks of the estimators is given.     

Estimation of the mean vector of a multivariate normal distribution: subspace hypothesis

This paper considers the estimation of the mean vector θ of a p-variate normal distribution with unknown covariance matrix Σ when it is suspected that for a p×r known matrix B the hypothesis θ=Bη, η∈R^r may hold. We consider empirical Bayes estimators which includes (i) the unrestricted unbiased (UE) estimator, namely, the sample mean vector (ii) the restricted estimator (RE) which is obtained when the hypothesis θ=Bη holds (iii) the preliminary test estimator (PTE), (iv) the James-Stein estimator (JSE), and (v) the positive-rule Stein estimator (PRSE). The biases and the risks under the squared loss function are evaluated for all the five estimators and compared. The numerical computations show that PRSE is the best among all the five estimators even when the hypothesis θ=Bη is true.     

Shrinkage estimation in the frequency domain of multivariate time series

In this paper on developing shrinkage for spectral analysis of multivariate time series of high dimensionality, we propose a new nonparametric estimator of the spectral matrix with two appealing properties. First, compared to the traditional smoothed periodogram our shrinkage estimator has a smaller L₂ risk. Second, the proposed shrinkage estimator is numerically more stable due to a smaller condition number. We use the concept of “Kolmogorov” asymptotics where simultaneously the sample size and the dimensionality tend to infinity, to show that the smoothed periodogram is not consistent and to derive the asymptotic properties of our regularized estimator. This estimator is shown to have asymptotically minimal risk among all linear combinations of the identity and the averaged periodogram matrix. Compared to existing work on shrinkage in the time domain, our results show that in the frequency domain it is necessary to take the size of the smoothing span as “effective sample size” into account. Furthermore, we perform extensive Monte Carlo studies showing the overwhelming gain in terms of lower L₂ risk of our shrinkage estimator, even in situations of oversmoothing the periodogram by using a large smoothing span.