Stein's phenomenon in estimation of means restricted to a polyhedral convex cone

This paper treats the problem of estimating the restricted means of normal distributions with a known variance, where the means are restricted to a polyhedral convex cone which includes various restrictions such as positive orthant, simple order, tree order and umbrella order restrictions. In the context of the simultaneous estimation of the restricted means, it is of great interest to investigate decision-theoretic properties of the generalized Bayes estimator against the uniform prior distribution over the polyhedral convex cone. In this paper, the generalized Bayes estimator is shown to be minimax. It is also proved that it is admissible in the one- or two-dimensional case, but is improved on by a shrinkage estimator in the three- or more-dimensional case. This means that the so-called Stein phenomenon on the minimax generalized Bayes estimator can be extended to the case where the means are restricted to the polyhedral convex cone. The risk behaviors of the estimators are investigated through Monte Carlo simulation, and it is revealed that the shrinkage estimator has a substantial risk reduction.     

Methods for improvement in estimation of a normal mean matrix

This paper is concerned with the problem of estimating a matrix of means in multivariate normal distributions with an unknown covariance matrix under invariant quadratic loss. It is first shown that the modified Efron-Morris estimator is characterized as a certain empirical Bayes estimator. This estimator modifies the crude Efron-Morris estimator by adding a scalar shrinkage term. It is next shown that the idea of this modification provides a general method for improvement of estimators, which results in the further improvement on several minimax estimators. As a new method for improvement, an adaptive combination of the modified Stein and the James-Stein estimators is also proposed and is shown to be minimax. Through Monte Carlo studies of the risk behaviors, it is numerically shown that the proposed, combined estimator inherits the nice risk properties of both individual estimators and thus it has a very favorable risk behavior in a small sample case. Finally, the application to a two-way layout MANOVA model with interactions is discussed.     

Shrinkage priors for Bayesian estimation of the mean matrix in an elliptically contoured distribution

This paper deals with the problem of estimating the mean matrix in an elliptically contoured distribution with unknown scale matrix. The Laplace and inverse Laplace transforms of the density allow us not only to evaluate the risk function with respect to a quadratic loss but also to simplify expressions of Bayes estimators. Consequently, it is shown that generalized Bayes estimators against shrinkage priors dominate the unbiased estimator.     

Modifying estimators of ordered positive parameters under the Stein loss

This paper treats the problem of estimating positive parameters restricted to a polyhedral convex cone which includes typical order restrictions, such as simple order, tree order and umbrella order restrictions. In this paper, two methods are used to show the improvement of order-preserving estimators over crude non-order-preserving estimators without any assumption on underlying distributions. One is to use Fenchel’s duality theorem, and then the superiority of the isotonic regression estimator is established under the general restriction to polyhedral convex cones. The use of the Abel identity is the other method, and we can derive a class of improved estimators which includes order-statistics-based estimators in the typical order restrictions. When the underlying distributions are scale families, the unbiased estimators and their order-restricted estimators are shown to be minimax. The minimaxity of the restrictedly generalized Bayes estimator against the prior over the restricted space is also demonstrated in the two dimensional case. Finally, some examples and multivariate extensions are given.     

Admissibility and minimaxity of generalized Bayes estimators for spherically symmetric family

We give a sufficient condition for admissibility of generalized Bayes estimators of the location vector of spherically symmetric distribution under squared error loss. Compared to the known results for the multivariate normal case, our sufficient condition is very tight and is close to being a necessary condition. In particular, we establish the admissibility of generalized Bayes estimators with respect to the harmonic prior and priors with slightly heavier tail than the harmonic prior. We use the theory of regularly varying functions to construct a sequence of smooth proper priors approaching an improper prior fast enough for establishing the admissibility. We also discuss conditions of minimaxity of the generalized Bayes estimator with respect to the harmonic prior.     

Simultaneous estimation of Poisson means

In the simultaneous estimation of means from independent Poisson distributions, an estimator is developed which incorporates a prior mean and variance for each Poisson mean estimated. This estimator possesses substantially smaller risk than the usual estimator in a region of the parameter space and seems superior to other estimators proposed to estimate p Poisson means. It is indicated through two asymptotic results that, unlike the conjugate Bayes estimator, the risk of the estimator does not greatly exceed the risk of the usual estimator outside of the region of risk improvement.     

On minimaxity and admissibility of hierarchical Bayes estimators

This paper obtains conditions for minimaxity of hierarchical Bayes estimators in the estimation of a mean vector of a multivariate normal distribution. Hierarchical prior distributions with three types of second stage priors are treated. Conditions for admissibility and inadmissibility of the hierarchical Bayes estimators are also derived using the arguments in Berger and Strawderman [Choice of hierarchical priors: admissibility in estimation of normal means, Ann. Statist. 24 (1996) 931-951]. Combining these results yields admissible and minimax hierarchical Bayes estimators.     

A subclass of bayes linear estimators that are minimax

In the linear regression model with ellipsoidal parameter constraints, the problem of estimating the unknown parameter vector is studied. A well-described subclass of Bayes linear estimators is proposed in the paper. It is shown that for each member of this subclass, a generalized quadratic risk function exists so that the estimator is minimax. Moreover, some of the proposed Bayes linear estimators are admissible with respect to all possible generalized quadratic risks. Also, a necessary and sufficient condition is given to ensure that the considered Bayes linear estimator improves the least squares estimator over the whole ellipsoid whatever generalized risk function is chosen.     

Estimation of the precision matrix of a singular Wishart distribution and its application in high-dimensional data

In this article, the Stein-Haff identity is established for a singular Wishart distribution with a positive definite mean matrix but with the dimension larger than the degrees of freedom. This identity is then used to obtain estimators of the precision matrix improving on the estimator based on the Moore-Penrose inverse of the Wishart matrix under the Efron-Morris loss function and its variants. Ridge-type empirical Bayes estimators of the precision matrix are also given and their dominance properties over the usual one are shown using this identity. Finally, these precision estimators are used in a quadratic discriminant rule, and it is shown through simulation that discriminant methods based on the ridge-type empirical Bayes estimators provide higher correct classification rates.     

Robust Hierarchical Bayes Estimation of Small Area Characteristics in the Presence of Covariates and Outliers

A robust hierarchical Bayes method is developed to smooth small area means when a number of covariates are available. The method is particularly suited when one or more outliers are present in the data. It is well known that the regular Bayes estimators of small. area means, under normal prior distribution, perform poorly in presence of even one extreme observation. In this case the Bayes estimators collapse to the direct survey estimators. This paper introduces a general theory for robust hierarchical Bayes estimation procedure using a fairly rich class of scale mixtures of normal prior distributions. To retain maximum benefit from combining information from related sources, we suggest to use Cauchy prior distribution for the outlying areas and an appropriate scale mixture of normal prior whose tail is lighter than the Cauchy prior for the rest of the areas. It is shown that, unlike the hierarchical Bayes estimator under a normal prior, our estimator has more protection against outlying observations.     

Estimation, prediction and the Stein phenomenon under divergence loss

We consider two problems: (1) estimate a normal mean under a general divergence loss introduced in [S. Amari, Differential geometry of curved exponential families — curvatures and information loss, Ann. Statist. 10 (1982) 357-387] and [N. Cressie, T.R.C. Read, Multinomial goodness-of-fit tests, J. Roy. Statist. Soc. Ser. B. 46 (1984) 440-464] and (2) find a predictive density of a new observation drawn independently of observations sampled from a normal distribution with the same mean but possibly with a different variance under the same loss. The general divergence loss includes as special cases both the Kullback-Leibler and Bhattacharyya-Hellinger losses. The sample mean, which is a Bayes estimator of the population mean under this loss and the improper uniform prior, is shown to be minimax in any arbitrary dimension. A counterpart of this result for predictive density is also proved in any arbitrary dimension. The admissibility of these rules holds in one dimension, and we conjecture that the result is true in two dimensions as well. However, the general Baranchick [A.J. Baranchick, a family of minimax estimators of the mean of a multivariate normal distribution, Ann. Math. Statist. 41 (1970) 642-645] class of estimators, which includes the James-Stein estimator and the Strawderman [W.E. Strawderman, Proper Bayes minimax estimators of the multivariate normal mean, Ann. Math. Statist. 42 (1971) 385-388] class of estimators, dominates the sample mean in three or higher dimensions for the estimation problem. An analogous class of predictive densities is defined and any member of this class is shown to dominate the predictive density corresponding to a uniform prior in three or higher dimensions. For the prediction problem, in the special case of Kullback-Leibler loss, our results complement to a certain extent some of the recent important work of Komaki [F. Komaki, A shrinkage predictive distribution for multivariate normal observations, Biometrika 88 (2001) 859-864] and George, Liang and Xu [E.I. George, F. Liang, X. Xu, Improved minimax predictive densities under Kullbak-Leibler loss, Ann. Statist. 34 (2006) 78-92]. While our proposed approach produces a general class of predictive densities (not necessarily Bayes, but not excluding Bayes predictors) dominating the predictive density under a uniform prior. We show also that various modifications of the James-Stein estimator continue to dominate the sample mean, and by the duality of estimation and predictive density results which we will show, similar results continue to hold for the prediction problem as well.     

Empirical Bayes predictive densities for high-dimensional normal models

This paper addresses the problem of estimating the density of a future outcome from a multivariate normal model. We propose a class of empirical Bayes predictive densities and evaluate their performances under the Kullback–Leibler (KL) divergence. We show that these empirical Bayes predictive densities dominate the Bayesian predictive density under the uniform prior and thus are minimax under some general conditions. We also establish the asymptotic optimality of these empirical Bayes predictive densities in infinite-dimensional parameter spaces through an oracle inequality.     

Generalized Bayes minimax estimators of the mean of multivariate normal distribution with unknown variance

We construct a broad class of generalized Bayes minimax estimators of the mean of a multivariate normal distribution with covariance equal to σ²I_p, with σ² unknown, and under the invariant loss δ(X)−θ²/σ². Examples that illustrate the theory are given. Most notably it is shown that a hierarchical version of the multivariate Student-t prior yields a Bayes minimax estimate.     

Minimax Bayes estimators of a multivariate normal mean

In three or more dimensions it is well known that the usual point estimator for the mean of a multivariate normal distribution is minimax but not admissible with respect to squared Euclidean distance loss. This paper gives sufficient conditions on the prior distribution under which the Bayes estimator has strictly lower risk than the usual estimator. Examples are given for which the posterior density is useful in the formation of confidence sets.     

Generalized Bayes minimax estimators of location vectors for spherically symmetric distributions

Let X∼f(∥x-θ∥²) and let δ_π(X) be the generalized Bayes estimator of θ with respect to a spherically symmetric prior, π(∥θ∥²), for loss ∥δ-θ∥². We show that if π(t) is superharmonic, non-increasing, and has a non-decreasing Laplacian, then the generalized Bayes estimator is minimax and dominates the usual minimax estimator δ₀(X)=X under certain conditions on . The class of priors includes priors of the form for and hence includes the fundamental harmonic prior . The class of sampling distributions includes certain variance mixtures of normals and other functions f(t) of the form e^-αtβ and e^-αt+βφ(t) which are not mixtures of normals. The proofs do not rely on boundness or monotonicity of the function r(t) in the representation of the Bayes estimator as .     

An adjusted maximum likelihood method for solving small area estimation problems

For the well-known Fay-Herriot small area model, standard variance component estimation methods frequently produce zero estimates of the strictly positive model variance. As a consequence, an empirical best linear unbiased predictor of a small area mean, commonly used in small area estimation, could reduce to a simple regression estimator, which typically has an overshrinking problem. We propose an adjusted maximum likelihood estimator of the model variance that maximizes an adjusted likelihood defined as a product of the model variance and a standard likelihood (e.g., a profile or residual likelihood) function. The adjustment factor was suggested earlier by Carl Morris in the context of approximating a hierarchical Bayes solution where the hyperparameters, including the model variance, are assumed to follow a prior distribution. Interestingly, the proposed adjustment does not affect the mean squared error property of the model variance estimator or the corresponding empirical best linear unbiased predictors of the small area means in a higher order asymptotic sense. However, as demonstrated in our simulation study, the proposed adjustment has a considerable advantage in small sample inference, especially in estimating the shrinkage parameters and in constructing the parametric bootstrap prediction intervals of the small area means, which require the use of a strictly positive consistent model variance estimate.     

Combining the data from two normal populations to estimate the mean of one when their means difference is bounded

In this paper we address the problem of estimating θ₁ when , are observed and |θ₁−θ₂|?c for a known constant c. Clearly Y₂ contains information about θ₁. We show how the so-called weighted likelihood function may be used to generate a class of estimators that exploit that information. We discuss how the weights in the weighted likelihood may be selected to successfully trade bias for precision and thus use the information effectively. In particular, we consider adaptively weighted likelihood estimators where the weights are selected using the data. One approach selects such weights in accord with Akaike's entropy maximization criterion. We describe several estimators obtained in this way. However, the maximum likelihood estimator is investigated as a competitor to these estimators along with a Bayes estimator, a class of robust Bayes estimators and (when c is sufficiently small), a minimax estimator. Moreover we will assess their properties both numerically and theoretically. Finally, we will see how all of these estimators may be viewed as adaptively weighted likelihood estimators. In fact, an over-riding theme of the paper is that the adaptively weighted likelihood method provides a powerful extension of its classical counterpart.     

A class of proper priors for Bayesian simultaneous prediction of independent Poisson observables

Simultaneous prediction and parameter inference for the independent Poisson observables model are considered. A class of proper prior distributions for Poisson means is introduced. Bayesian predictive densities and estimators based on priors in the introduced class dominate the Bayesian predictive density and estimator based on the Jeffreys prior under Kullback-Leibler loss.     

Bayes minimax estimators of the mean of a scale mixture of multivariate normal distributions

Bayes estimation of the mean of a variance mixture of multivariate normal distributions is considered under sum of squared errors loss. We find broad class of priors (also in the variance mixture of normal class) which result in proper and generalized Bayes minimax estimators. This paper extends the results of Strawderman [Minimax estimation of location parameters for certain spherically symmetric distribution, J. Multivariate Anal. 4 (1974) 255-264] in a manner similar to that of Maruyama [Admissible minimax estimators of a mean vector of scale mixtures of multivariate normal distribution, J. Multivariate Anal. 21 (2003) 69-78] but somewhat more in the spirit of Fourdrinier et al. [On the construction of bayes minimax estimators, Ann. Statist. 26 (1998) 660-671] for the normal case, in the sense that we construct classes of priors giving rise to minimaxity. A feature of this paper is that in certain cases we are able to construct proper Bayes minimax estimators satisfying the properties and bounds in Strawderman [Minimax estimation of location parameters for certain spherically symmetric distribution, J. Multivariate Anal. 4 (1974) 255-264]. We also give some insight into why Strawderman's results do or do not seem to apply in certain cases. In cases where it does not apply, we give minimax estimators based on Berger's [Minimax estimation of location vectors for a wide class of densities, Ann. Statist. 3 (1975) 1318-1328] results. A main condition for minimaxity is that the mixing distributions of the sampling distribution and the prior distribution satisfy a monotone likelihood ratio property with respect to a scale parameter.     

Estimation of means of multivariate normal populations with order restriction

Multivariate isotonic regression theory plays a key role in the field of statistical inference under order restriction for vector valued parameters. Two cases of estimating multivariate normal means under order restricted set are considered. One case is that covariance matrices are known, the other one is that covariance matrices are unknown but are restricted by partial order. This paper shows that when covariance matrices are known, the estimator given by this paper always dominates unrestricted maximum likelihood estimator uniformly, and when covariance matrices are unknown, the plug-in estimator dominates unrestricted maximum likelihood estimator under the order restricted set of covariance matrices. The isotonic regression estimators in this paper are the generalizations of plug-in estimators in unitary case.