Minimax estimation of the mean of spherically symmetric distributions under general quadratic loss

For X one observation on a p-dimensional (p ≥ 4) spherically symmetric (s.s.) distribution about θ, minimax estimators whose risks dominate the risk of X (the best invariant procedure) are found with respect to general quadratic loss, L(δ, θ) = (δ − θ)′ D(δ − θ) where D is a known p × p positive definite matrix. For C a p × p known positive definite matrix, conditions are given under which estimators of the form δ_a,r,C,D(X) = (I − (ar(|X|²)) D^−1/2CD^1/2 |X|⁻²)X are minimax with smaller risk than X. For the problem of estimating the mean when n observations X₁, X₂, …, X_n are taken on a p-dimensional s.s. distribution about θ, any spherically symmetric translation invariant estimator, δ(X₁, X₂, …, X_n), with have a s.s. distribution about θ. Among the estimators which have these properties are best invariant estimators, sample means and maximum likelihood estimators. Moreover, under certain conditions, improved robust estimators can be found.     

Estimation of a Normal Covariance Matrix with Incomplete Data under Stein′s Loss

Suppose that we have (n − a) independent observations from N_p(0, Σ) and that, in addition, we have a independent observations available on the last (p − c) coordinates. Assuming that both observations are independent, we consider the problem of estimating Σ under the Stein′s loss function, and show that some estimators invariant under the permutation of the last (p − c) coordinates as well as under those of the first c coordinates are better than the minimax estimators of Eaten. The estimators considered outperform the maximum likelihood estimator (MLE) under the Stein′s loss function as well. The method involved here is computation of an unbiased estimate of the risk of an invariant estimator considered in this article. In addition we discuss its application to the problem of estimating a covariance matrix in a GMANOVA model since the estimation problem of the covariance matrix with extra data can be regarded as its canonical form.     

Generalized Hodges-Lehmann estimators for the analysis of variance

For X_i, …, X_n a random sample and K(·, ·) a symmetric kernel this paper considers large sample properties of location estimator satisfying , . Asymptotic normality of is obtained and two forms of interval estimators for parameter θ satisfying EK(X₁ − θ, X₂ − θ) = 0, are discussed. Consistent estimation of the variance parameters is obtained which permits the construction of asymptotically distribution free procedures. The p-variate and multigroup extension is accomplished to provide generalized one-way MANOVA. Monte Carlo results are included.     

Admissibility of the natural estimator of the mean of a Gaussian process

Let {X(t): t [a, b]} be a Gaussian process with mean μ L₂[a, b] and continuous covariance K(s, t). When estimating μ under the loss ∫_a^b ( (t)−μ(t))² dt the natural estimator X is admissible if K is unknown. If K is known, X is minimax with risk ∫_a^b K(t, t) dt and admissible if and only if the three by three matrix whose entries are K(t_i, t_j) has a determinant which vanishes identically in t_i [a, b], i = 1, 2, 3.     

On improved estimators of the generalized variance

Treated in this paper is the problem of estimating with squared error loss the generalized variance | Σ | from a Wishart random matrix S: p × p W_p(n, Σ) and an independent normal random matrix X: p × k N(ξ, Σ I_k) with ξ(p × k) unknown. Denote the columns of X by X₍₁₎ ,…, X_(k) and set ψ⁽⁰⁾(S, X) = {(n − p + 2)!/(n + 2)!} | S |, ψ⁽ⁱ⁾(X, X) = min[ψ⁽ⁱ⁻¹⁾(S, X), {(n − p + i + 2)!/(n + i + 2)!} | S + X₍₁₎ X′₍₁₎ + + X_(i) X′_(i) |] and Ψ⁽ⁱ⁾(S, X) = min[ψ⁽⁰⁾(S, X), {(n − p + i + 2)!/(n + i + 2)!}| S + X₍₁₎ X′₍₁₎ + + X_(i) X′_(i) |], i = 1,…,k. Our result is that the minimax, best affine equivariant estimator ψ⁽⁰⁾(S, X) is dominated by each of Ψ⁽ⁱ⁾(S, X), i = 1,…,k and for every i, ψ⁽ⁱ⁾(S, X) is better than ψ⁽ⁱ⁻¹⁾(S, X). In particular, ψ^(k)(S, X) = min[{(n − p + 2)!/(n + 2)!} | S |, {(n − p + 2)!/(n + 2)!} | S + X₍₁₎X′₍₁₎|,…,| {(n − p + k + 2)!/(n + k + 2)!} | S + X₍₁₎X′₍₁₎ + + X_(k)X′_(k)|] dominates all other ψ's. It is obtained by considering a multivariate extension of Stein's result (Ann. Inst. Statist. Math. 16, 155–160 (1964)) on the estimation of the normal variance.     

Some extensions of the Kantorovich inequality and statistical applications

Kantorovich gave an upper bound to the product of two quadratic forms, (X′AX) (X′A⁻¹X), where X is an n-vector of unit length and A is a positive definite matrix. Bloomfield, Watson and Knott found the bound for the product of determinants |X′AX| |X′A⁻¹X| where X is n × k matrix such that X′X = I_k. In this paper we determine the bounds for the traces and determinants of matrices of the type X′AYY′A⁻¹X, X′B²X(X′BCX)⁻¹ X′C²X(X′BCX)⁻¹ where X and Y are n × k matrices such that X′X = Y′Y = I_k and A, B, C are given matrices satisfying some conditions. The results are applied to the least squares theory of estimation.     

On confidence sequences for the mean vector of a multivariate normal distribution

Let X₁, X₂,… be idd random vectors with a multivariate normal distribution N(μ, Σ). A sequence of subsets {R_n(a₁, a₂,…, a_n), n ≥ m} of the space of μ is said to be a (1 − α)-level sequence of confidence sets for μ if P(μ R_n(X₁, X₂,…, X_n) for every n ≥ m) ≥ 1 − α. In this note we use the ideas of Robbins Ann. Math. Statist. 41 (1970) to construct confidence sequences for the mean vector μ when Σ is either known or unknown. The constructed sequence R_n(X₁, X₂, …, X_n) depends on Mahalanobis' or Hotelling's according as Σ is known or unknown. Confidence sequences for the vector-valued parameter in the general linear model are also given.     

On the approximate behavior of the posterior distribution for an extreme multivariate observation

The behavior of the posterior for a large observation is considered. Two basic situations are discussed; location vectors and natural parameters.Let X = (X₁, X₂, …, X_n) be an observation from a multivariate exponential distribution with that natural parameter Θ = (Θ₁, Θ₂, …, Θ_n). Let θ_x^* be the posterior mode. Sufficient conditions are presented for the distribution of Θ − θ_x^* given X = x to converge to a multivariate normal with mean vector 0 as |x| tends to infinity. These same conditions imply that E(Θ | X = x) − θ_x^* converges to the zero vector as |x| tends to infinity.The posterior for an observation X = (X₁, X₂, …, X_n is considered for a location vector Θ = (Θ₁, Θ₂, …, Θ_n) as x gets large along a path, γ, in Rⁿ. Sufficient conditions are given for the distribution of γ(t) − Θ given X = γ(t) to converge in law as t → ∞. Slightly stronger conditions ensure that γ(t) − E(Θ | X = γ(t)) converges to the mean of the limiting distribution.These basic results about the posterior mean are extended to cover other estimators. Loss functions which are convex functions of absolute error are considered. Let δ be a Bayes estimator for a loss function of this type. Generally, if the distribution of Θ − E(Θ | X = γ(t)) given X = γ(t) converges in law to a symmetric distribution as t → ∞, it is shown that δ(γ(t)) − E(Θ | X = γ(t)) → 0 as t → ∞.     

On Bayes and unbiased estimators of loss

We consider estimation of loss for generalized Bayes or pseudo-Bayes estimators of a multivariate normal mean vector, θ. In 3 and higher dimensions, the MLEX is UMVUE and minimax but is inadmissible. It is dominated by the James-Stein estimator and by many others. Johnstone (1988, On inadmissibility of some unbiased estimates of loss,Statistical Decision Theory and Related Topics, IV (eds. S. S. Gupta and J. O. Berger), Vol. 1, 361–379, Springer, New York) considered the estimation of loss for the usual estimatorX and the James-Stein estimator. He found improvements over the Stein unbiased estimator of risk. In this paper, for a generalized Bayes point estimator of θ, we compare generalized Bayes estimators to unbiased estimators of loss. We find, somewhat surprisingly, that the unbiased estimator often dominates the corresponding generalized Bayes estimator of loss for priors which give minimax estimators in the original point estimation problem. In particular, we give a class of priors for which the generalized Bayes estimator of θ is admissible and minimax but for which the unbiased estimator of loss dominates the generalized Bayes estimator of loss. We also give a general inadmissibility result for a generalized Bayes estimator of loss. Research supported by NSF Grant DMS-97-04524.     

Admissibility and complete class results for the multinomial estimation problem with entropy and squared error loss

Let X ≡ (X₁, …, X_t) have a multinomial distribution based on N trials with unknown vector of cell probabilities p ≡ (p₁, …, p_t). This paper derives admissibility and complete class results for the problem of simultaneously estimating p under entropy loss (EL) and squared error loss (SEL). Let and f(x¦p) denote the (t − 1)-dimensional simplex, the support of X and the probability mass function of X, respectively. First it is shown that δ is Bayes w.r.t. EL for prior P if and only if δ is Bayes w.r.t. SEL for P. The admissible rules under EL are proved to be Bayes, a result known for the case of SEL. Let Q denote the class of subsets of of the form T = _j=1^kF_j where k ≥ 1 and each F_j is a facet of which satisfies: F a facet of such that F naF_jF ncT. The minimal complete class of rules w.r.t. EL when N ≥ t − 1 is characterized as the class of Bayes rules with respect to priors P which satisfy P( ⁰) = 1, ξ(x) ≡ ∫ f(x¦p) P(dp) > 0 for all x in {x : sup ^₀ f(x¦p) > 0} for some ⁰ in Q containing all the vertices of . As an application, the maximum likelihood estimator is proved to be admissible w.r.t. EL when the estimation problem has parameter space Θ = but it is shown to be inadmissible for the problem with parameter space Θ = ( minus its vertices). This is a severe form of “tyranny of boundary.” Finally it is shown that when N ≥ t − 1 any estimator δ which satisfies δ(x) > 0 x is admissible under EL if and only if it is admissible under SEL. Examples are given of nonpositive estimators which are admissible under SEL but not under EL and vice versa.     

A family of minimax estimators of a multivariate normal mean

Let X be a p-dimensional normal random vector with unknown mean vector θ and covariance σ²I. Let S/σ², independent of X, be chi-square with n degrees of freedom. Relative to the squared error loss, James and Stein (1961) have obtained an estimator which dominates the usual estimator X. Baranchik (1970) has extended James and Stein's results. We obtain a theorem which can provide a different family of minimax estimators containing James-Stein's estimator. Two interesting minimax estimators are presented in this paper.     

Admissibility under the Frequentist′s Validity Constraint in Estimating the Loss of the Least-Squares Estimator

We consider the problem of estimating the sum of squared error loss L = |β− |² of the least-squares esitmator for β, the regression coefficient. The standard estimator ₀ is the expected value of L. Here the error variance is assumed to be known. Previous results of Johnstone (1988. In Statistical Decision Theory and Related Topics IV (S. Gupta and J. Berger, Eds.), 1, 361-379, Springer-Verlag, New York) show that ₀ is inadmissible under the loss ( −L)² if the dimension of is five or more. However, since we are estimating the loss, a typical frequentist principle will lead to the usage of estimators which are frequentist valid. Johnston′s improved esitmator, however, violates this principle. In this paper, we prove that it is impossible to improve upon ₀ among the class of frequentist valid estimators. The work parallels Hwang and Brown (1991, Ann. Statist.10 1964-1977) for the corresponding confidence set problems, although the argument is entirely different and much simpler.     

An asymptotic minimax risk bound for estimation of a linear functional relationship

We consider estimation of the parameter B in a multivariate linear functional relationship X_i=ξ_i+ξ_1i, Y_i=Bξ_i+ξ_2i, i=1,…,n, where the errors (ζ_1i^′, ζ_2i^′) are independent standard normal and (ξ_i, i ) is a sequence of unknown nonrandom vectors (incidental parameters). If there are no substantial a priori restrictions on the infinite sequence of incidental parameters then asymptotically the model is nonparametric but does not fit into common settings presupposing a parameter from a metric function space. A special result of the local asymptotic minimax type for the m.1.e. of B is proved. The accuracy of the normal approximation for the m.l.e. of order n^−1/2 is also established.     

Stochastic comparisons of order statistics from heterogeneous populations, with applications in reliability

The basic result of the paper states: Let F₁, …, F_n, F₁^′,…, F_n^′ have proportional hazard functions with λ₁ ,…, λ_n , λ₁^′ ,…, λ_n^′ as the constants of proportionality. Let X₍₁₎ ≤ … ≤ X_(n) (X₍₁₎^′ ≤ … ≤ X_(n)^′) be the order statistics in a sample of size n from the heterogeneous populations {F₁ ,…, F_n}({F₁^′ ,…, F_n^′}). Then (λ₁ ,…, λ_n) majorizes (λ₁^′ ,…, λ_n^′) implies that (X₍₁₎ ,…, X_(n)) is stochastically larger than (X₍₁₎^′ ,…, X_(n)^′). Earlier results stochastically comparing individual order statistics are shown to be special cases. Applications of the main result are made in the study of the robustness of standard estimates of the failure rate of the exponential distribution, when observations actually come from a set of heterogeneous exponential distributions. Further applications are made to the comparisons of linear combinations of Weibull random variables and of binomial random variables.     

Generalized Bayes minimax estimators of the mean of multivariate normal distribution with unknown variance

We construct a broad class of generalized Bayes minimax estimators of the mean of a multivariate normal distribution with covariance equal to σ²I_p, with σ² unknown, and under the invariant loss δ(X)−θ²/σ². Examples that illustrate the theory are given. Most notably it is shown that a hierarchical version of the multivariate Student-t prior yields a Bayes minimax estimate.     

Some properties of BLUE in a linear model and canonical correlations associated with linear transformations

Let (x, Xβ, V) be a linear model and let A′ = (A′₁, A′₂) be a p × p nonsingular matrix such that A₂X = 0, Rank A₂ = p − Rank X. We represent the BLUE and its covariance matrix in alternative forms under the conditions that the number of unit canonical correlations between y₁ ( = A₁x) and y₂ ( = A₂x) is zero. For the second problem, let x′ = (x′₁, x′₂) and let a g-inverse V⁻ of V be written as (V⁻)′ = (A′₁, A′₂). We investigate the reations (if any) between the nonzero canonical correlations {1 ₁ … ₁ > 0} due to y₁ ( = A₁x) and y₂ ( = A₂x), and the nonzero canonical correlations {1 λ₁ … λ_v+r > 0} due to x₁ and x₂. We answer some of the questions raised by Latour et al. (1987, in Proceedings, 2nd Int. Tampere Conf. Statist. (T. Pukkila and S. Puntanen, Eds.), Univ. of Tampere, Finland) in the case of the Moore-Penrose inverse V⁺ = (A′₁, A′₂) of V.     

Estimating the mean function of a Gaussian process and the Stein effect

The problem of global estimation of the mean function θ(·) of a quite arbitrary Gaussian process is considered. The loss function in estimating θ by a function a(·) is assumed to be of the form L(θ, a) = ∫ [θ(t) ? a(t)]²μ(dt), and estimators are evaluated in terms of their risk function (expected loss). The usual minimax estimator of θ is shown to be inadmissible via the Stein phenomenon; in estimating the function θ we are trying to simultaneously estimate a larger number of normal means. Estimators improving upon the usual minimax estimator are constructed, including an estimator which allows the incorporation of prior information about θ. The analysis is carried out by using a version of the Karhunen-Loéve expansion to represent the original problem as the problem of estimating a countably infinite sequence of means from independent normal distributions.     

Estimates of Freud-Christoffel functions for some weights with the whole real line as support

Upper and lower bounds for generalized Christoffel functions, called Freud-Christoffel functions, are obtained. These have the form λ_n,p(W,j,x) = infPW_Lp(R)/|P^(j)(X)| where the infimum is taken over all polynomials P(x) of degree at most n − 1. The upper and lower bounds for λ_n,p(W,j,x) are obtained for all 0 < p ∞ and J = 0, 1, 2, 3,… for weights W(x) = exp(−Q(x)), where, among other things, Q(x) is bounded in [− A, A], and Q″ is continuous in β(−A, A) for some A > 0. For p = ∞, the lower bounds give a simple proof of local and global Markov-Bernstein inequalities. For p = 2, the results remove some restrictions on Q in Freud's work. The weights considered include W(x) = exp(− ¦x¦^α/2), α > 0, and W(x) = exp(− exp(¦x¦)), > 0.     

Maximum likelihood estimation of a change-point in the distribution of independent random variables: General multiparameter case

In a sequence ofn independent random variables the pdf changes fromf(x, 0) tof(x, 0 + δv_n⁻¹) after the firstnλ variables. The problem is to estimateλ (0, 1 ), where 0 and δ are unknownd-dim parameters andv_n → ∞ slower thann^1/2. Let_n denote the maximum likelihood estimator (mle) ofλ. Analyzing the local behavior of the likelihood function near the true parameter values it is shown under regularity conditions that ifn_n²(− λ) is bounded in probability asn → ∞, then it converges in law to the timeT_(δjδ)^1/2 at which a two-sided Brownian motion (B.M.) with drift1/2(δ′Jδ)^1/2ton(−∞, ∞) attains its a.s. unique minimum, whereJ denotes the Fisher-information matrix. This generalizes the result for small change in mean of univariate normal random variables obtained by Bhattacharya and Brockwell (1976,Z. Warsch. Verw. Gebiete37, 51–75) who also derived the distribution ofT_μ forμ > 0. For the general case an alternative estimator is constructed by a three-step procedure which is shown to have the above asymptotic distribution. In the important case of multiparameter exponential families, the construction of this estimator is considerably simplified.     

I.i.d. representations for the bivariate product limit estimators and the bootstrap versions

Let F(s, t) = P(X > s, Y > t) be the bivariate survival function which is subject to random censoring. Let be the bivariate product limit estimator (PL-estimator) by Campbell and Földes (1982, Proceedings International Colloquium on Non-parametric Statistical Inference, Budapest 1980, North-Holland, Amsterdam). In this paper, it was shown that

Full-size image

, where {ζ_i(s, t)} is i.i.d. mean zero process and R_n(s, t) is of the order O((n⁻¹log n)^3/4) a.s. uniformly on compact sets. Weak convergence of the process {n⁻¹ Σ_{i = 1}ⁿ ζ_i(s, t)} to a two-dimensional-time Gaussian process is shown. The covariance structure of the limiting Gaussian process is also given. Corresponding results are also derived for the bootstrap estimators. The result can be extended to the multivariate cases and are extensions of the univariate case of Lo and Singh (1986, Probab. Theory Relat. Fields, 71, 455–465). The estimator is also modified so that the modified estimator is closer to the true survival function than in supnorm.