Minimax estimation of means of multivariate normal mixtures

Assume X = (X₁, …, X_p)′ is a normal mixture distribution with density w.r.t. Lebesgue measure, , where Σ is a known positive definite matrix and F is any known c.d.f. on (0, ∞). Estimation of the mean vector under an arbitrary known quadratic loss function _Q(θ, a) = (a − θ)′ Q(a − θ), Q a positive definite matrix, is considered. An unbiased estimator of risk is obatined for an arbitrary estimator, and a sufficient condition for estimators to be minimax is then achieved. The result is applied to modifying all the Stein estimators for the means of independent normal random variables to be minimax estimators for the problem considered here. In particular the results apply to the Stein class of limited translation estimators.     

On improved estimators of the generalized variance

Treated in this paper is the problem of estimating with squared error loss the generalized variance | Σ | from a Wishart random matrix S: p × p W_p(n, Σ) and an independent normal random matrix X: p × k N(ξ, Σ I_k) with ξ(p × k) unknown. Denote the columns of X by X₍₁₎ ,…, X_(k) and set ψ⁽⁰⁾(S, X) = {(n − p + 2)!/(n + 2)!} | S |, ψ⁽ⁱ⁾(X, X) = min[ψ⁽ⁱ⁻¹⁾(S, X), {(n − p + i + 2)!/(n + i + 2)!} | S + X₍₁₎ X′₍₁₎ + + X_(i) X′_(i) |] and Ψ⁽ⁱ⁾(S, X) = min[ψ⁽⁰⁾(S, X), {(n − p + i + 2)!/(n + i + 2)!}| S + X₍₁₎ X′₍₁₎ + + X_(i) X′_(i) |], i = 1,…,k. Our result is that the minimax, best affine equivariant estimator ψ⁽⁰⁾(S, X) is dominated by each of Ψ⁽ⁱ⁾(S, X), i = 1,…,k and for every i, ψ⁽ⁱ⁾(S, X) is better than ψ⁽ⁱ⁻¹⁾(S, X). In particular, ψ^(k)(S, X) = min[{(n − p + 2)!/(n + 2)!} | S |, {(n − p + 2)!/(n + 2)!} | S + X₍₁₎X′₍₁₎|,…,| {(n − p + k + 2)!/(n + k + 2)!} | S + X₍₁₎X′₍₁₎ + + X_(k)X′_(k)|] dominates all other ψ's. It is obtained by considering a multivariate extension of Stein's result (Ann. Inst. Statist. Math. 16, 155–160 (1964)) on the estimation of the normal variance.     

Generalized Hodges-Lehmann estimators for the analysis of variance

For X_i, …, X_n a random sample and K(·, ·) a symmetric kernel this paper considers large sample properties of location estimator satisfying , . Asymptotic normality of is obtained and two forms of interval estimators for parameter θ satisfying EK(X₁ − θ, X₂ − θ) = 0, are discussed. Consistent estimation of the variance parameters is obtained which permits the construction of asymptotically distribution free procedures. The p-variate and multigroup extension is accomplished to provide generalized one-way MANOVA. Monte Carlo results are included.     

On the approximate behavior of the posterior distribution for an extreme multivariate observation

The behavior of the posterior for a large observation is considered. Two basic situations are discussed; location vectors and natural parameters.Let X = (X₁, X₂, …, X_n) be an observation from a multivariate exponential distribution with that natural parameter Θ = (Θ₁, Θ₂, …, Θ_n). Let θ_x^* be the posterior mode. Sufficient conditions are presented for the distribution of Θ − θ_x^* given X = x to converge to a multivariate normal with mean vector 0 as |x| tends to infinity. These same conditions imply that E(Θ | X = x) − θ_x^* converges to the zero vector as |x| tends to infinity.The posterior for an observation X = (X₁, X₂, …, X_n is considered for a location vector Θ = (Θ₁, Θ₂, …, Θ_n) as x gets large along a path, γ, in Rⁿ. Sufficient conditions are given for the distribution of γ(t) − Θ given X = γ(t) to converge in law as t → ∞. Slightly stronger conditions ensure that γ(t) − E(Θ | X = γ(t)) converges to the mean of the limiting distribution.These basic results about the posterior mean are extended to cover other estimators. Loss functions which are convex functions of absolute error are considered. Let δ be a Bayes estimator for a loss function of this type. Generally, if the distribution of Θ − E(Θ | X = γ(t)) given X = γ(t) converges in law to a symmetric distribution as t → ∞, it is shown that δ(γ(t)) − E(Θ | X = γ(t)) → 0 as t → ∞.     

Generalized Bayes minimax estimators of the mean of multivariate normal distribution with unknown variance

We construct a broad class of generalized Bayes minimax estimators of the mean of a multivariate normal distribution with covariance equal to σ²I_p, with σ² unknown, and under the invariant loss δ(X)−θ²/σ². Examples that illustrate the theory are given. Most notably it is shown that a hierarchical version of the multivariate Student-t prior yields a Bayes minimax estimate.     

Robust M-estimators of location vectors

Let X₁,…,X_n be i.i.d. random vectors in R^m, let θεR^m be an unknown location parameter, and assume that the restriction of the distribution of X₁−θ to a sphere of radius d belongs to a specified neighborhood of distributions spherically symmetric about 0. Under regularity conditions on and d, the parameter θ in this model is identifiable, and consistent M-estimators of θ (i.e., solutions of Σ_i=1ⁿψ(|X_i− |)(X_i− )=0) are obtained by using “re-descenders,” i.e., ψ's wh satisfy ψ(x)=0 for x≥c. An iterative method for solving for is shown to produce consistent and asymptotically normal estimates of θ under all distributions in . The following asymptotic robustness problem is considered: finding the ψ which is best among the re-descenders according to Huber's minimax variance criterion.     

Global nonparametric estimation of conditional quantile functions and their derivatives

Let (X, Y) be a random vector such that X is d-dimensional, Y is real valued, and θ(X) is the conditional αth quantile of Y given X, where α is a fixed number such that 0 < α < 1. Assume that θ is a smooth function with order of smoothness p > 0, and set r = (p − m)/(2p + d), where m is a nonnegative integer smaller than p. Let T(θ) denote a derivative of θ of order m. It is proved that there exists estimate of T(θ), based on a set of i.i.d. observations (X₁, Y₁), …, (X_n, Y_n), that achieves the optimal nonparametric rate of convergence n^−r in L_q-norms (1 ≤ q < ∞) restricted to compacts under appropriate regularity conditions. Further, it has been shown that there exists estimate of T(θ) that achieves the optimal rate (n/log n)^−r in L_∞-norm restricted to compacts.     

Admissibility and complete class results for the multinomial estimation problem with entropy and squared error loss

Let X ≡ (X₁, …, X_t) have a multinomial distribution based on N trials with unknown vector of cell probabilities p ≡ (p₁, …, p_t). This paper derives admissibility and complete class results for the problem of simultaneously estimating p under entropy loss (EL) and squared error loss (SEL). Let and f(x¦p) denote the (t − 1)-dimensional simplex, the support of X and the probability mass function of X, respectively. First it is shown that δ is Bayes w.r.t. EL for prior P if and only if δ is Bayes w.r.t. SEL for P. The admissible rules under EL are proved to be Bayes, a result known for the case of SEL. Let Q denote the class of subsets of of the form T = _j=1^kF_j where k ≥ 1 and each F_j is a facet of which satisfies: F a facet of such that F naF_jF ncT. The minimal complete class of rules w.r.t. EL when N ≥ t − 1 is characterized as the class of Bayes rules with respect to priors P which satisfy P( ⁰) = 1, ξ(x) ≡ ∫ f(x¦p) P(dp) > 0 for all x in {x : sup ^₀ f(x¦p) > 0} for some ⁰ in Q containing all the vertices of . As an application, the maximum likelihood estimator is proved to be admissible w.r.t. EL when the estimation problem has parameter space Θ = but it is shown to be inadmissible for the problem with parameter space Θ = ( minus its vertices). This is a severe form of “tyranny of boundary.” Finally it is shown that when N ≥ t − 1 any estimator δ which satisfies δ(x) > 0 x is admissible under EL if and only if it is admissible under SEL. Examples are given of nonpositive estimators which are admissible under SEL but not under EL and vice versa.     

Schur functions and the invariant polynomials characterizing U(n) tensor operators

We give a direct formulation of the invariant polynomials _μG_q⁽ⁿ⁾(, Δ_i,;, x_{i,i + 1},) characterizing U(n) tensor operators p, q, …, q, 0, …, 0 in terms of the symmetric functions S_λ known as Schur functions. To this end, we show after the change of variables Δ_i = γ_i − δ_i and x_{i, i + 1} = δ_i − δ_{i + 1} that_μG_q⁽ⁿ⁾(,Δ_i;, x_{i, i + 1},) becomes an integral linear combination of products of Schur functions S_α(, γ_i,) · S_β(, δ_i,) in the variables {γ₁,…, γ_n} and {δ₁,…, δ_n}, respectively. That is, we give a direct proof that _μG_q⁽ⁿ⁾(,Δ_i,;, x_{i, i + 1},) is a bisymmetric polynomial with integer coefficients in the variables {γ₁,…, γ_n} and {δ₁,…, δ_n}. By making further use of basic properties of Schur functions such as the Littlewood-Richardson rule, we prove several remarkable new symmetries for the yet more general bisymmetric polynomials _μ^mG_q⁽ⁿ⁾(γ₁,…, γ_n; δ₁,…, δ_m). These new symmetries enable us to give an explicit formula for both _μ^mG₁⁽ⁿ⁾(γ; δ) and ₁G₂⁽ⁿ⁾(γ; δ). In addition, we describe both algebraic and numerical integration methods for deriving general polynomial formulas for _μ^mG_q⁽ⁿ⁾(γ; δ).     

Stochastic comparisons of order statistics from heterogeneous populations, with applications in reliability

The basic result of the paper states: Let F₁, …, F_n, F₁^′,…, F_n^′ have proportional hazard functions with λ₁ ,…, λ_n , λ₁^′ ,…, λ_n^′ as the constants of proportionality. Let X₍₁₎ ≤ … ≤ X_(n) (X₍₁₎^′ ≤ … ≤ X_(n)^′) be the order statistics in a sample of size n from the heterogeneous populations {F₁ ,…, F_n}({F₁^′ ,…, F_n^′}). Then (λ₁ ,…, λ_n) majorizes (λ₁^′ ,…, λ_n^′) implies that (X₍₁₎ ,…, X_(n)) is stochastically larger than (X₍₁₎^′ ,…, X_(n)^′). Earlier results stochastically comparing individual order statistics are shown to be special cases. Applications of the main result are made in the study of the robustness of standard estimates of the failure rate of the exponential distribution, when observations actually come from a set of heterogeneous exponential distributions. Further applications are made to the comparisons of linear combinations of Weibull random variables and of binomial random variables.     

On confidence sequences for the mean vector of a multivariate normal distribution

Let X₁, X₂,… be idd random vectors with a multivariate normal distribution N(μ, Σ). A sequence of subsets {R_n(a₁, a₂,…, a_n), n ≥ m} of the space of μ is said to be a (1 − α)-level sequence of confidence sets for μ if P(μ R_n(X₁, X₂,…, X_n) for every n ≥ m) ≥ 1 − α. In this note we use the ideas of Robbins Ann. Math. Statist. 41 (1970) to construct confidence sequences for the mean vector μ when Σ is either known or unknown. The constructed sequence R_n(X₁, X₂, …, X_n) depends on Mahalanobis' or Hotelling's according as Σ is known or unknown. Confidence sequences for the vector-valued parameter in the general linear model are also given.     

Estimation of a Normal Covariance Matrix with Incomplete Data under Stein′s Loss

Suppose that we have (n − a) independent observations from N_p(0, Σ) and that, in addition, we have a independent observations available on the last (p − c) coordinates. Assuming that both observations are independent, we consider the problem of estimating Σ under the Stein′s loss function, and show that some estimators invariant under the permutation of the last (p − c) coordinates as well as under those of the first c coordinates are better than the minimax estimators of Eaten. The estimators considered outperform the maximum likelihood estimator (MLE) under the Stein′s loss function as well. The method involved here is computation of an unbiased estimate of the risk of an invariant estimator considered in this article. In addition we discuss its application to the problem of estimating a covariance matrix in a GMANOVA model since the estimation problem of the covariance matrix with extra data can be regarded as its canonical form.     

Prophet regions and sharp inequalities for pth absolute moments of martingales

Exact comparisons are made relating E|Y₀|^p, E|Y_n−1|^p, and E(max_j≤n−1 |Y_j|^p), valid for all martingales Y₀,…,Y_n−1, for each p ≥ 1. Specifically, for p > 1, the set of ordered triples {(x, y, z) : X = E|Y₀|^p, Y = E |Y_n−1|^p, and Z = E(max_j≤n−1 |Y_j|^p) for some martingale Y₀,…,Y_n−1} is precisely the set {(x, y, z) : 0≤x≤y≤z≤Ψ_n,p(x, y)}, where Ψ_n,p(x, y) = xψ_n,p(y/x) if x > 0, and = a_n−1,py if x = 0; here ψ_n,p is a specific recursively defined function. The result yields families of sharp inequalities, such as E(max_j≤n−1 |Y_j|^p) + ψ_n,p^*(a) E |Y₀|^p ≤ aE |Y_n−1|^p, valid for all martingales Y₀,…,Y_n−1, where ψ_n,p^* is the concave conjugate function of ψ_n,p. Both the finite sequence and infinite sequence cases are developed. Proofs utilize moment theory, induction, conjugate function theory, and functional equation analysis.     

Estimates of Freud-Christoffel functions for some weights with the whole real line as support

Upper and lower bounds for generalized Christoffel functions, called Freud-Christoffel functions, are obtained. These have the form λ_n,p(W,j,x) = infPW_Lp(R)/|P^(j)(X)| where the infimum is taken over all polynomials P(x) of degree at most n − 1. The upper and lower bounds for λ_n,p(W,j,x) are obtained for all 0 < p ∞ and J = 0, 1, 2, 3,… for weights W(x) = exp(−Q(x)), where, among other things, Q(x) is bounded in [− A, A], and Q″ is continuous in β(−A, A) for some A > 0. For p = ∞, the lower bounds give a simple proof of local and global Markov-Bernstein inequalities. For p = 2, the results remove some restrictions on Q in Freud's work. The weights considered include W(x) = exp(− ¦x¦^α/2), α > 0, and W(x) = exp(− exp(¦x¦)), > 0.     

Consistency of a recursive nearest neighbor regression function estimate

Let (X, Y) be an ^d × -valued random vector and let (X₁, Y₁),…,(X_N, Y_N) be a random sample drawn from its distribution. Divide the data sequence into disjoint blocks of length l₁, …, l_n, find the nearest neighbor to X in each block and call the corresponding couple (X_i^*, Y_i^*). It is shown that the estimate m_n(X) = Σ_{i = 1}ⁿ w_niY_i^*/Σ_{i = 1}ⁿ w_ni of m(X) = E{Y|X} satisfies E{|m_n(X) − m(X)|^p} 0 (p ≥ 1) whenever E{|Y|^p} < ∞, l_n ∞, and the triangular array of positive weights {w_ni} satisfies sup_{i ≤ n}w_ni/Σ_{i = 1}ⁿ w_ni 0. No other restrictions are put on the distribution of (X, Y). Also, some distribution-free results for the strong convergence of E{|m_n(X) − m(X)|^p|X₁, Y₁,…, X_N, Y_N} to zero are included. Finally, an application to the discrimination problem is considered, and a discrimination rule is exhibited and shown to be strongly Bayes risk consistent for all distributions.     

MU-Estimation and Smoothing

In the M-estimation theory developed by Huber (1964, Ann. Math. Statist.43, 1449–1458), the parameter under estimation is the value of θ which minimizes the expectation of what is called a discrepancy measure (DM) δ(X, θ) which is a function of θ and the underlying random variable X. Such a setting does not cover the estimation of parameters such as the multivariate median defined by Oja (1983) and Liu (1990), as the value of θ which minimizes the expectation of a DM of the type δ(X₁, …, X_m, θ) where X₁, …, X_m are independent copies of the underlying random variable X. Arcones et al. (1994, Ann. Statist.22, 1460–1477) studied the estimation of such parameters. We call such an M-type MU-estimation (or μ-estimation for convenience). When a DM is not a differentiable function of θ, some complexities arise in studying the properties of estimators as well as in their computation. In such a case, we introduce a new method of smoothing the DM with a kernel function and using it in estimation. It is seen that smoothing allows us to develop an elegant approach to the study of asymptotic properties and possibly apply the Newton–Raphson procedure in the computation of estimators.     

Asymptotic expansions in the conditional central limit theorem

Let X_n, n , be i.i.d. with mean 0, variance 1, and E(¦X_n¦^r) < ∞ for some r 3. Assume that Cramér's condition is fulfilled. We prove that the conditional probabilities P(1/√n Σ_{i = 1}ⁿ X_i t¦B) can be approximated by a modified Edgeworth expansion up to order o(1/n^{(r − 2)/2)}), if the distances of the set B from the σ-fields σ(X₁, …, X_n) are of order O(1/n^{(r − 2)/2)}(lg n)^β), where β < −(r − 2)/2 for r and β < −r/2 for r . An example shows that if we replace β < −(r − 2)/2 by β = −(r − 2)/2 for r (β < −r/2 by β = −r/2 for r ) we can only obtain the approximation order O(1/n^{(r − 2)/2)}) for r (O(lg lgn/n^{(r − 2)/2)}) for r ).     

A uniform bound for the deviation of empirical distribution functions

If X₁, …, X_n are independent R^d-valued random vectors with common distribution function F, and if F_n is the empirical distribution function for X₁, …, X_n, then, among other things, it is shown that P{sup_x F_n(x) ε} 2e²(2n)^de^−2nε2 for all nε² ≥ d². The inequality remains valid if the X_i are not identically distributed and F(x) is replaced by Σ_iP{X_i ≤ x}/n.     

A discrete Korovkin theorem

In this paper we give a sufficient condition for the pointwise Korovkin property on B(X), the space of bounded real valued functions on an arbitrary countable set X = {x_l,…, x_j,…}. Our theorem follows from its L_p(X, μ) analogue (and conversely); here 1 p < ∞ and μ is a positive finite measure on X such that μ({x_j}) > 0 for all j.     

On the structure of biharmonic functions satisfying the clamped plate conditions on a right angle

Let u(r,θ) be biharmonic and bounded in the circular sector ¦θ¦ < π/4, 0 < r < ρ (ρ > 1) and vanish together with δu/δθ when ¦θ¦ = π/4. We consider the transform û(p,θ) = ∝₀¹r^{p − 1}u(r,θ)dr. We show that for any fixed θ₀ u(p,θ₀) is meromorphic with no real poles and cannot be entire unless u(r, θ₀) ≡ 0. It follows then from a theorem of Doetsch that u(r, θ₀) either vanishes identically or oscillates as r → 0.