Improving on the mle of a bounded location parameter for spherical distributions

For the problem of estimating under squared error loss the location parameter of a p-variate spherically symmetric distribution where the location parameter lies in a ball of radius m, a general sufficient condition for an estimator to dominate the maximum likelihood estimator is obtained. Dominance results are then made explicit for the case of a multivariate student distribution with d degrees of freedom and, in particular, we show that the Bayes estimator with respect to a uniform prior on the boundary of the parameter space dominates the maximum likelihood estimator whenever and d?p. The sufficient condition matches the one obtained by Marchand and Perron (Ann. Statist. 29 (2001) 1078) in the normal case with identity covariance matrix. Furthermore, we derive an explicit class of estimators which, for , dominate the maximum likelihood estimator simultaneously for the normal distribution with identity covariance matrix and for all multivariate student distributions with d degrees of freedom, d?p. Finally, we obtain estimators which dominate the maximum likelihood estimator simultaneously for all distributions in the subclass of scale mixtures of normals for which the scaling random variable is bounded below by some positive constant with probability one.     

On Bayes estimators with uniform priors on spheres and their comparative performance with maximum likelihood estimators for estimating bounded multivariate normal means

For independently distributed observables: X_i∼N(θ_i,σ²),i=1,…,p, we consider estimating the vector θ=(θ₁,…,θ_p)^′ with loss ‖d−θ‖² under the constraint , with known τ₁,…,τ_p,σ²,m. In comparing the risk performance of Bayesian estimators δ_α associated with uniform priors on spheres of radius α centered at (τ₁,…,τ_p) with that of the maximum likelihood estimator , we make use of Stein’s unbiased estimate of risk technique, Karlin’s sign change arguments, and a conditional risk analysis to obtain for a fixed (m,p) necessary and sufficient conditions on α for δ_α to dominate . Large sample determinations of these conditions are provided. Both cases where all such δ_α’s and cases where no such δ_α’s dominate are elicited. We establish, as a particular case, that the boundary uniform Bayes estimator δ_m dominates if and only if m≤k(p) with , improving on the previously known sufficient condition of Marchand and Perron (2001) [3] for which . Finally, we improve upon a universal dominance condition due to Marchand and Perron, by establishing that all Bayesian estimators δ_π with π spherically symmetric and supported on the parameter space dominate whenever m≤c₁(p) with .     

Asymptotic properties of Bayes estimators for Gaussian Itô-processes with noisy observations

The estimation of a real parameter θ in a linear stochastic differential equation of the simple type is investigated, based on noisy, time continuous observations of X_t. Sufficient conditions on the continuous functions β and σ are given such that the (conditionally normal) Bayes estimators of θ satisfy certain error bounds and are strongly consistent.     

Generalized Bayes minimax estimators of location vectors for spherically symmetric distributions

Let X∼f(∥x-θ∥²) and let δ_π(X) be the generalized Bayes estimator of θ with respect to a spherically symmetric prior, π(∥θ∥²), for loss ∥δ-θ∥². We show that if π(t) is superharmonic, non-increasing, and has a non-decreasing Laplacian, then the generalized Bayes estimator is minimax and dominates the usual minimax estimator δ₀(X)=X under certain conditions on . The class of priors includes priors of the form for and hence includes the fundamental harmonic prior . The class of sampling distributions includes certain variance mixtures of normals and other functions f(t) of the form e^-αtβ and e^-αt+βφ(t) which are not mixtures of normals. The proofs do not rely on boundness or monotonicity of the function r(t) in the representation of the Bayes estimator as .     

Estimation of location parameters for spherically symmetric distributions

Estimation of the location parameters of a p×1 random vector with a spherically symmetric distribution is considered under quadratic loss. The conditions of Brandwein and Strawderman [Ann. Statist. 19(1991) 1639-1650] under which estimators of the form dominate are (i) where -h is superharmonic, (ii) is nonincreasing in R, where has a uniform distribution in the sphere centered at with a radius R, and (iii) . In this paper, we not only drop their condition (ii) to show the dominance of over but also obtain a new bound for a which is sometimes better than that obtained by Brandwein and Strawderman. Specifically, the new bound of a is 0<a<[μ₁/(p²μ_-1)][1-(p-1)μ₁/(pμ_-1μ₂)]^-1 with for i=-1,1,2. The generalization to concave loss functions is also considered. Additionally, we investigate estimators of the location parameters when the scale is unknown and the observation contains a residual vector.     

The Stein phenomenon for monotone incomplete multivariate normal data

We establish the Stein phenomenon in the context of two-step, monotone incomplete data drawn from , a (p+q)-dimensional multivariate normal population with mean and covariance matrix . On the basis of data consisting of n observations on all p+q characteristics and an additional N−n observations on the last q characteristics, where all observations are mutually independent, denote by the maximum likelihood estimator of . We establish criteria which imply that shrinkage estimators of James-Stein type have lower risk than under Euclidean quadratic loss. Further, we show that the corresponding positive-part estimators have lower risk than their unrestricted counterparts, thereby rendering the latter estimators inadmissible. We derive results for the case in which is block-diagonal, the loss function is quadratic and non-spherical, and the shrinkage estimator is constructed by means of a nondecreasing, differentiable function of a quadratic form in . For the problem of shrinking to a vector whose components have a common value constructed from the data, we derive improved shrinkage estimators and again determine conditions under which the positive-part analogs have lower risk than their unrestricted counterparts.     

Robust semiparametric M-estimation and the weighted bootstrap

M-estimation is a widely used technique for statistical inference. In this paper, we study properties of ordinary and weighted M-estimators for semiparametric models, especially when there exist parameters that cannot be estimated at the convergence rate. Results on consistency, rates of convergence for all parameters, and consistency and asymptotic normality for the Euclidean parameters are provided. These results, together with a generic paradigm for studying semiparametric M-estimators, provide a valuable extension to previous related research on semiparametric maximum-likelihood estimators (MLEs). Although penalized M-estimation does not in general fit in the framework we discuss here, it is shown for a great variety of models that many of the forgoing results still hold, including the consistency and asymptotic normality of the Euclidean parameters. For semiparametric M-estimators that are not likelihood based, general inference procedures for the Euclidean parameters have not previously been developed. We demonstrate that our paradigm leads naturally to verification of the validity of the weighted bootstrap in this setting. For illustration, several examples are investigated in detail. The new M-estimation framework and accompanying weighted bootstrap technique shed light on a universal way of investigating semiparametric models.     

Confidence intervals for marginal parameters under fractional linear regression imputation for missing data

Item nonresponse occurs frequently in sample surveys and other applications. Imputation is commonly used to fill in the missing item values in a random sample {Y_i;i=1,…,n}. Fractional linear regression imputation, based on the model with independent zero mean errors ?_i, is used to create one or more imputed values in the data file for each missing item Y_i, where {X_i,i=1,…,n}, is observed completely. Asymptotic normality of the imputed estimators of the mean μ=E(Y), distribution function θ=F(y) for a given y, and qth quantile θ_q=F^-1(q),0<q<1 is established, assuming that Y is missing at random (MAR) given X. This result is used to obtain normal approximation (NA)-based confidence intervals on μ,θ and θ_q. In the case of θ_q, a Bahadur-type representation and Woodruff-type confidence intervals are also obtained. Empirical likelihood (EL) ratios are also obtained and shown to be asymptotically scaled variables. This result is used to obtain asymptotically correct EL-based confidence intervals on μ,θ and θ_q. Results of a simulation study on the finite sample performance of NA-based and EL-based confidence intervals are reported.     

Adaptive estimation of the transition density of a particular hidden Markov chain

We study the following model of hidden Markov chain: with (X_i) a real-valued positive recurrent and stationary Markov chain, and (?_i)_1?i?n+1 a noise independent of the sequence (X_i) having a known distribution. We present an adaptive estimator of the transition density based on the quotient of a deconvolution estimator of the density of X_i and an estimator of the density of (X_i,X_i+1). These estimators are obtained by contrast minimization and model selection. We evaluate the L² risk and its rate of convergence for ordinary smooth and supersmooth noise with regard to ordinary smooth and supersmooth chains. Some examples are also detailed.     

Statistical inference of minimum BD estimators and classifiers for varying-dimensional models

Stochastic modeling for large-scale datasets usually involves a varying-dimensional model space. This paper investigates the asymptotic properties, when the number of parameters grows with the available sample size, of the minimum- estimators and classifiers under a broad and important class of Bregman divergence (), which encompasses nearly all of the commonly used loss functions in the regression analysis, classification procedures and machine learning literature. Unlike the maximum likelihood estimators which require the joint likelihood of observations, the minimum-BD estimators are useful for a range of models where the joint likelihood is unavailable or incomplete. Statistical inference tools developed for the class of large dimensional minimum- estimators and related classifiers are evaluated via simulation studies, and are illustrated by analysis of a real dataset.     

Nonparametric comparison of regression functions

In this work, we provide a new methodology for comparing regression functions m₁ and m₂ from two samples. Since apart from smoothness no other (parametric) assumptions are required, our approach is based on a comparison of nonparametric estimators and of m₁ and m₂, respectively. The test statistics incorporate weighted differences of and computed at selected points. Since the design variables may come from different distributions, a crucial question is where to compare the two estimators. As our main results we obtain the limit distribution of (properly standardized) under the null hypothesis H₀:m₁=m₂ and under local and global alternatives. We are also able to choose the weight function so as to maximize the power. Furthermore, the tests are asymptotically distribution free under H₀ and both shift and scale invariant. Several such ’s may then be combined to get Maximin tests when the dimension of the local alternative is finite. In a simulation study we found out that our tests achieve the nominal level and already have excellent power for small to moderate sample sizes.     

Estimation of the mean vector of a multivariate normal distribution: subspace hypothesis

This paper considers the estimation of the mean vector θ of a p-variate normal distribution with unknown covariance matrix Σ when it is suspected that for a p×r known matrix B the hypothesis θ=Bη, η∈R^r may hold. We consider empirical Bayes estimators which includes (i) the unrestricted unbiased (UE) estimator, namely, the sample mean vector (ii) the restricted estimator (RE) which is obtained when the hypothesis θ=Bη holds (iii) the preliminary test estimator (PTE), (iv) the James-Stein estimator (JSE), and (v) the positive-rule Stein estimator (PRSE). The biases and the risks under the squared loss function are evaluated for all the five estimators and compared. The numerical computations show that PRSE is the best among all the five estimators even when the hypothesis θ=Bη is true.