A comparison of minimax and least squares estimators in linear regression with polyhedral prior information

We consider the linear regression model where prior information in the form of linear inequalities restricts the parameter space to a polyhedron. Since the linear minimax estimator has, in general, to be determined numerically, it was proposed to minimize an upper bound of the maximum risk instead. The resulting so-called quasiminimax estimator can be easily calculated in closed form. Unfortunately, both minimax estimators may violate the prior information. Therefore, we consider projection estimators which are obtained by projecting the estimate in an optional second step. The performance of these estimators is investigated in a Monte Carlo study together with several least squares estimators, including the inequality restricted least squares estimator. It turns out that both the projected and the unprojected quasiminimax estimators have the best average performance.     

Thresholding projection estimators in functional linear models

We consider the problem of estimating the regression function in functional linear regression models by proposing a new type of projection estimators which combine dimension reduction and thresholding. The introduction of a threshold rule allows us to get consistency under broad assumptions as well as minimax rates of convergence under additional regularity hypotheses. We also consider the particular case of Sobolev spaces generated by the trigonometric basis which permits us to get easily mean squared error of prediction as well as estimators of the derivatives of the regression function. We prove that these estimators are minimax and rates of convergence are given for some particular cases.     

Generalized Bayes minimax estimators of location vectors for spherically symmetric distributions

Let X∼f(∥x-θ∥²) and let δ_π(X) be the generalized Bayes estimator of θ with respect to a spherically symmetric prior, π(∥θ∥²), for loss ∥δ-θ∥². We show that if π(t) is superharmonic, non-increasing, and has a non-decreasing Laplacian, then the generalized Bayes estimator is minimax and dominates the usual minimax estimator δ₀(X)=X under certain conditions on . The class of priors includes priors of the form for and hence includes the fundamental harmonic prior . The class of sampling distributions includes certain variance mixtures of normals and other functions f(t) of the form e^-αtβ and e^-αt+βφ(t) which are not mixtures of normals. The proofs do not rely on boundness or monotonicity of the function r(t) in the representation of the Bayes estimator as .     

Generalized Bayes minimax estimators of the mean of multivariate normal distribution with unknown variance

We construct a broad class of generalized Bayes minimax estimators of the mean of a multivariate normal distribution with covariance equal to σ²I_p, with σ² unknown, and under the invariant loss δ(X)−θ²/σ². Examples that illustrate the theory are given. Most notably it is shown that a hierarchical version of the multivariate Student-t prior yields a Bayes minimax estimate.     

Bayes minimax estimators of the mean of a scale mixture of multivariate normal distributions

Bayes estimation of the mean of a variance mixture of multivariate normal distributions is considered under sum of squared errors loss. We find broad class of priors (also in the variance mixture of normal class) which result in proper and generalized Bayes minimax estimators. This paper extends the results of Strawderman [Minimax estimation of location parameters for certain spherically symmetric distribution, J. Multivariate Anal. 4 (1974) 255-264] in a manner similar to that of Maruyama [Admissible minimax estimators of a mean vector of scale mixtures of multivariate normal distribution, J. Multivariate Anal. 21 (2003) 69-78] but somewhat more in the spirit of Fourdrinier et al. [On the construction of bayes minimax estimators, Ann. Statist. 26 (1998) 660-671] for the normal case, in the sense that we construct classes of priors giving rise to minimaxity. A feature of this paper is that in certain cases we are able to construct proper Bayes minimax estimators satisfying the properties and bounds in Strawderman [Minimax estimation of location parameters for certain spherically symmetric distribution, J. Multivariate Anal. 4 (1974) 255-264]. We also give some insight into why Strawderman's results do or do not seem to apply in certain cases. In cases where it does not apply, we give minimax estimators based on Berger's [Minimax estimation of location vectors for a wide class of densities, Ann. Statist. 3 (1975) 1318-1328] results. A main condition for minimaxity is that the mixing distributions of the sampling distribution and the prior distribution satisfy a monotone likelihood ratio property with respect to a scale parameter.     

Depth estimators and tests based on the likelihood principle with application to regression

We investigate depth notions for general models which are derived via the likelihood principle. We show that the so-called likelihood depth for regression in generalized linear models coincides with the regression depth of Rousseeuw and Hubert (J. Amer. Statist. Assoc. 94 (1999) 388) if the dependent observations are appropriately transformed. For deriving tests, the likelihood depth is extended to simplicial likelihood depth. The simplicial likelihood depth is always a U-statistic which is in some cases not degenerated. Since the U-statistic is degenerated in the most cases, we demonstrate that nevertheless the asymptotic distribution of the simplicial likelihood depth and thus asymptotic α-level tests for general types of hypotheses can be derived. The tests are distribution-free. We work out the method for linear and quadratic regression.     

Admissibility and minimaxity of Bayes estimators for a normal mean matrix

In some invariant estimation problems under a group, the Bayes estimator against an invariant prior has equivariance as well. This is useful notably for evaluating the frequentist risk of the Bayes estimator. This paper addresses the problem of estimating a matrix of means in normal distributions relative to quadratic loss. It is shown that a matricial shrinkage Bayes estimator against an orthogonally invariant hierarchical prior is admissible and minimax by means of equivariance. The analytical improvement upon every over-shrinkage equivariant estimator is also considered and this paper justifies the corresponding positive-part estimator preserving the order of the sample singular values.     

A unified and generalized set of shrinkage bounds on minimax Stein estimates

Consider the problem of estimating the mean vector θ of a random variable X in , with a spherically symmetric density f(x−θ²), under loss δ−θ². We give an increasing sequence of bounds on the shrinkage constant of Stein-type estimators depending on properties of f(t) that unify and extend several classical bounds from the literature. The basic way to view the conditions on f(t) is that the distribution of X arises as the projection of a spherically symmetric vector (X,U) in . A second way is that f(t) satisfies (−1)^jf^(j)(t)≥0 for 0≤j≤ℓ and that (−1)^ℓf^(ℓ)(t) is non-increasing where k=2(ℓ+1). The case ℓ=0 (k=2) corresponds to unimodality, while the case ℓ=k=∞ corresponds to complete monotonicity of f(t) (or equivalently that f(x−θ²) is a scale mixture of normals). The bounds on the minimax shrinkage constant in this paper agree with the classical bounds in the literature for the case of spherical symmetry, spherical symmetry and unimodality, and scale mixtures of normals. However, they extend these bounds to an increasing sequence (in k or ℓ) of minimax bounds.     

Nonnegative quadratic estimation of the mean squared errors of minimax estimators in the linear regression model

In this paper, the problem of nonnegative quadratic estimation of the mean squared errors of minimax estimators of in the linear regression modelE(y)=X, VAR(y) = ² is discussed. An explicit formula for the admissible nonnegative minimum biased estimator is given. Some applications to one-way classification model are also considered.     

Third-order power comparisons for a class of tests for multivariate linear hypothesis under general distributions

The purpose of this paper is, in multivariate linear regression model (Part I) and GMANOVA model (Part II), to investigate the effect of nonnormality upon the nonnull distributions of some multivariate test statistics under normality. It is shown that whatever the underlying distributions, the difference of local powers up to order N⁻¹ after either Bartlett’s type adjustment or Cornish-Fisher’s type size adjustment under nonnormality coincides with that in Anderson [An Introduction to Multivariate Statistical Analysis, 2nd ed. and 3rd ed., Wiley, New York, 1984, 2003] under normality. The derivation of asymptotic expansions is based on the differential operator associated with the multivariate linear regression model under general distributions. The performance of higher-order results in finite samples, including monotone Bartlett’s type adjustment and monotone Cornish-Fisher’s type size adjustment, is examined using simulation studies.     

Spectral methods in linear minimax estimation

We consider the minimax-linear estimator in a linear regression model with circular constraints. Two necessary and sufficient conditions for the optimality of an estimator, the socalled left spectral equation and the right spectral equation (Girko spectral equation), are derived. For the special case of a simple maximal eigenvalue and a single eigenspace explicit estimation formulas are derived. These formulas also show some of the shortcomings of the minimax-linear estimator (MILE). Finally, the relation with Bayesian analysis and the Hoffmann-Läuter estimator is outlined.     

Second order optimality for estimators in time series regression models

We consider the second order asymptotic properties of an efficient frequency domain regression coefficient estimator proposed by Hannan [Regression for time series, Proc. Sympos. Time Series Analysis (Brown Univ., 1962), Wiley, New York, 1963, pp. 17-37]. This estimator is a semiparametric estimator based on nonparametric spectral estimators. We derive the second order Edgeworth expansion of the distribution of . Then it is shown that the second order asymptotic properties are independent of the bandwidth choice for residual spectral estimator, which implies that has the same rate of convergence as in regular parametric estimation. This is a sharp contrast with the general semiparametric estimation theory. We also examine the second order Gaussian efficiency of . Numerical studies are given to confirm the theoretical results.     

Conditional and unconditional methods for selecting variables in linear mixed models

In the problem of selecting the explanatory variables in the linear mixed model, we address the derivation of the (unconditional or marginal) Akaike information criterion (AIC) and the conditional AIC (cAIC). The covariance matrices of the random effects and the error terms include unknown parameters like variance components, and the selection procedures proposed in the literature are limited to the cases where the parameters are known or partly unknown. In this paper, AIC and cAIC are extended to the situation where the parameters are completely unknown and they are estimated by the general consistent estimators including the maximum likelihood (ML), the restricted maximum likelihood (REML) and other unbiased estimators. We derive, related to AIC and cAIC, the marginal and the conditional prediction error criteria which select superior models in light of minimizing the prediction errors relative to quadratic loss functions. Finally, numerical performances of the proposed selection procedures are investigated through simulation studies.     

Minimax adjustment technique in a parameter restricted linear model

We consider an approach yielding a minimax estimator in the linear regression model with a priori information on the parameter vector, e.g., ellipsoidal restrictions. This estimator is computed directly from the loss function and can be motivated by the general Pitman nearness criterion. It turns out that this approach coincides with the projection estimator which is obtained by projecting an initial arbitrary estimate on the subset defined by the restrictions.     

Admissibility of linear estimators with respect to inequality constraints under matrix loss function

In this paper we investigate the admissibility of linear estimators in the multivariate linear model with respect to inequality constraints under matrix loss function. The necessary and sufficient conditions for a linear estimator to be admissible in the class of homogeneous linear estimators and the class of inhomogeneous linear estimators are obtained, respectively.     

Proper bayes minimax estimators for a multivariate normal mean with unknown common variance under a convex loss function

Summary Let X ∼ N_p(μ,σ²I_p) and let s/σ² ∼ χ _n ² , independent ofX, where μ and σ² are unknown. This paper considers the estimation of μ (by δ) relative to a convex loss function given by (δ−μ)′[(1−α)I_p/σ²+αQ](δ−μ)/[(1−α)p/σ²+α tr (Q)], whereQ is a knownp×p diagonal matrix and 0≦α≦1. Two classes of minimax estimators are obtained for μ whenp≦3; the first is a new result and the second is a generalization of a result of Strawderman (1973,Ann. Statist.,1, 1189–1194). A proper Bayes estimator is also obtained which is shown to satisfy the conditions of the second class of minimax estimators. The paper concludes by discussing the estimation of μ relative to another convex loss function. This work was supported by the Army, Navy and Air Force under Office of Naval Research Contract No. N00014-80-C-0093. Reproduction in whole or in part is permitted for any purpose of the United States Government.     

Modifying estimators of ordered positive parameters under the Stein loss

This paper treats the problem of estimating positive parameters restricted to a polyhedral convex cone which includes typical order restrictions, such as simple order, tree order and umbrella order restrictions. In this paper, two methods are used to show the improvement of order-preserving estimators over crude non-order-preserving estimators without any assumption on underlying distributions. One is to use Fenchel’s duality theorem, and then the superiority of the isotonic regression estimator is established under the general restriction to polyhedral convex cones. The use of the Abel identity is the other method, and we can derive a class of improved estimators which includes order-statistics-based estimators in the typical order restrictions. When the underlying distributions are scale families, the unbiased estimators and their order-restricted estimators are shown to be minimax. The minimaxity of the restrictedly generalized Bayes estimator against the prior over the restricted space is also demonstrated in the two dimensional case. Finally, some examples and multivariate extensions are given.     

Minimax least squares and quasiminimax estimation in linear models with missing values

We consider the problem of estimating the parameter vector in the linear model when observations on the independent variables are partially missing or incorrect. New estimators are developed, which systematically combine prior information with the incomplete data. We compare these methods with the alternative strategy of deleting incomplete observations.Support by Deutsche Forschungsgemeinschaft, Grant No. 284/1-2 is gratefully acknowledged.     

Linear plus quadratic (LPQ) quasiminimax estimation in the linear regression model

If the errors in the linear regression model are assumed to be independent with nonvanishing third and finite fourth moments, then it is possible to improve all linear estimators by so-called linear plus quadratic (LPQ) estimators. These consist of linear and quadratic terms in the endogeneous variable and depend on the unknown moments of the errors which, in general, have to be estimated from the data. In this paper, we will use LPQ estimators for quasiminimax estimation and some related problems.Support by Deutsche Forschungsgemeinschaft Grant No. Tr 253/1-2 is gratefully acknowledged.     

A decomposition for a stochastic matrix with an application to MANOVA

The aim of this paper is to propose a simple method in order to evaluate the (approximate) distribution of matrix quadratic forms when Wishartness conditions do not hold. The method is based upon a factorization of a general Gaussian stochastic matrix as a special linear combination of nonstochastic matrices with the standard Gaussian matrix. An application of previous result is proposed for matrix quadratic forms arising in MANOVA for a multivariate split-plot design with circular dependence structure.