On Bayes and unbiased estimators of loss

We consider estimation of loss for generalized Bayes or pseudo-Bayes estimators of a multivariate normal mean vector, θ. In 3 and higher dimensions, the MLEX is UMVUE and minimax but is inadmissible. It is dominated by the James-Stein estimator and by many others. Johnstone (1988, On inadmissibility of some unbiased estimates of loss,Statistical Decision Theory and Related Topics, IV (eds. S. S. Gupta and J. O. Berger), Vol. 1, 361–379, Springer, New York) considered the estimation of loss for the usual estimatorX and the James-Stein estimator. He found improvements over the Stein unbiased estimator of risk. In this paper, for a generalized Bayes point estimator of θ, we compare generalized Bayes estimators to unbiased estimators of loss. We find, somewhat surprisingly, that the unbiased estimator often dominates the corresponding generalized Bayes estimator of loss for priors which give minimax estimators in the original point estimation problem. In particular, we give a class of priors for which the generalized Bayes estimator of θ is admissible and minimax but for which the unbiased estimator of loss dominates the generalized Bayes estimator of loss. We also give a general inadmissibility result for a generalized Bayes estimator of loss. Research supported by NSF Grant DMS-97-04524.     

On the lower bound in second order estimation for Poisson processes: Asymptotic efficiency

In the estimation problem of the mean function of an inhomogeneous Poisson process there is a class of kernel type estimators that are asymptotically efficient alongside with the empirical mean function. We start by describing such a class of estimators which we call first order efficient estimators. To choose the best one among them we prove a lower bound that compares the second order term of the mean integrated square error of all estimators. The proof is carried out under the assumption on the mean function Λ(·) that Λ(τ) = S, where S is a known positive number. In the end, we discuss the possibility of the construction of an estimator which attains this lower bound, thus, is asymptotically second order efficient.     

Third order efficiency implies fourth order efficiency: A resolution of the conjecture of J. K. Ghosh

Under suitable regularity conditions, it is shown that a third order asymptotically efficient estimator is fourth order asymptotically efficient in some class of estimators in the sense that the estimator has the most concentration probability in any symmetric interval around the true parameter up to the fourth order in the class. This is a resolution of the conjecture by Ghosh (1994, Higher Order Asymptotics, Institute of Mathematical Statistics, Hayward, California). It is also shown that the bias-adjusted maximum likelihood estimator is fourth order asymptotically efficient in the class.     

Empirical Bayes estimation in a multiple linear regression model

Summary Estimation of the vector β of the regression coefficients in a multiple linear regressionY=Xβ+ε is considered when β has a completely unknown and unspecified distribution and the error-vector ε has a multivariate standard normal distribution. The optimal estimator for β, which minimizes the overall mean squared error, cannot be constructed for use in practice. UsingX, Y and the information contained in the observation-vectors obtained fromn independent past experiences of the problem, (empirical Bayes) estimators for β are exhibited. These estimators are compared with the optimal estimator and are shown to be asymptotically optimal. Estimators asymptotically optimal with rates nearO(n ⁻¹) are constructed. Supported in part by a Natural Sciences and Engineering Research Council of Canada grant.     

Minimum disparity estimation for continuous models: Efficiency,distributions and robustness

A general class of minimum distance estimators for continuous models called minimum disparity estimators are introduced. The conventional technique is to minimize a distance between a kernel density estimator and the model density. A new approach is introduced here in which the model and the data are smoothed with the same kernel. This makes the methods consistent and asymptotically normal independently of the value of the smoothing parameter; convergence properties of the kernel density estimate are no longer necessary. All the minimum distance estimators considered are shown to be first order efficient provided the kernel is chosen appropriately. Different minimum disparity estimators are compared based on their characterizing residual adjustment function (RAF); this function shows that the robustness features of the estimators can be explained by the shrinkage of certain residuals towards zero. The value of the second derivative of theRAF at zero,A ₂, provides the trade-off between efficiency and robustness. The above properties are demonstrated both by theorems and by simulations.     

Third order asymptotic properties of maximum likelihood estimators for Gaussian ARMA processes

In this paper we investigate various third-order asymptotic properties of maximum likelihood estimators for Gaussian ARMA processes by the third-order Edgeworth expansions of the sampling distributions. We define a third-order asymptotic efficiency by the highest probability concentration around the true value with respect to the third-order Edgeworth expansion. Then we show that the maximum likelihood estimator is not always third-order asymptotically efficient in the class A₃ of third-order asymptotically median unbiased estimators. But, if we confine our discussions to an appropriate class D (⊂ A₃) of estimators, we can show that appropriately modified maximum likelihood estimator is always third-order asymptotically efficient in D.     

Second order asymptotic optimality of estimators for a density with finite cusps

We consider i.i.d. samples from a continuous density with finite cusps. Then we obtain the bound for the second order asymptotic distribution of all asymptotically median unbiased estimators. Further we get the second order asymptotic distribution of a bias-adjusted maximum likelihood estimator, and we see that it is not generally second order asymptotically efficient.     

Validity of Edgeworth expansions of minimum contrast estimators for Gaussian ARMA processes

Let {X_t} be a Gaussian ARMA process with spectral density f_θ(λ), where θ is an unknown parameter. To estimate θ we propose a minimum contrast estimation method which includes the maximum likelihood method and the quasi-maximum likelihood method as special cases. Let θ̂_τ be the minimum contrast estimator of θ. Then we derive the Edgewroth expansion of the distribution of θ̂_τ up to third order, and prove its valldity. By this Edgeworth expansion we can see that this minimum contrast estimator is always second-order asymptotically efficient in the class of second-order asymptotically median unbiased estimators. Also the third-order asymptotic comparisons among minimum contrast estimators will be discussed.     

Asymptotic properties of the Bayes modal estimators of item parameters in item response theory

Asymptotic cumulants of the Bayes modal estimators of item parameters using marginal likelihood in item response theory are derived up to the fourth order with added higher-order asymptotic variances under possible model misspecification. Among them, only the first asymptotic cumulant and the higher-order asymptotic variance for an estimator are different from those by maximum likelihood. Corresponding results for studentized Bayes estimators and asymptotically bias-corrected ones are also obtained. It was found that all the asymptotic cumulants of the bias-corrected Bayes estimator up to the fourth order and the higher-order asymptotic variance are identical to those by maximum likelihood with bias correction. Numerical illustrations are given with simulations in the case when the 2-parameter logistic model holds. In the numerical illustrations, the maximum likelihood and Bayes estimators are used, where the same independent log-normal priors are employed for discriminant parameters and the hierarchical model is adopted for the prior of difficulty parameters.     

Asymptotically Efficient Estimation of the Derivative of the Invariant Density

The problem of estimation of the derivative of the invariant density is considered for a one-dimensional ergodic diffusion process. The lower minimax bound on the L ²-type risk of all estimators is proposed and an asymptotically efficient (up to the constant) in the sense of this bound kernel-type estimator is constructed.     

Consistent nonparametric regression from recursive partitioning schemes

We here extend our results on asymptotically Bayes risk efficient classification to the general regression scenario. More precisely, we find L^p consistent estimators for an arbitrary regression function provided only that the dependent variable has a finite absolute pth moment. The estimators are truncated and untruncated local means derived from recursive partitioning schemes.     

On the asymptotic efficiency of estimators in an autoregressive process

Summary Let {X _t } be defined recursively byX _t =θX _t−1+U _t (t=1,2, ...), whereX ₀=0 and {U _t } is a sequence of independent identically distributed real random variables having a density functionf with mean 0 and varianceσ ². We assume that |θ|<1. In the present paper we obtain the bound of the asymptotic distributions of asymptotically median unbiased (AMU) estimators of θ and the sufficient condition that an AMU estimator be asymptotically efficient in the sense that its distribution attains the above bound. It is also shown that the least squares estimator of θ is asymptotically efficient if and only iff is a normal density function. University of Electro-Communications     

Necessary conditions for dominating the James-Stein estimator

This paper develops necessary conditions for an estimator to dominate the James-Stein estimator and hence the James-Stein positive-part estimator. The ultimate goal is to find classes of such dominating estimators which are admissible. While there are a number of results giving classes of estimators dominating the James-Stein estimator, the only admissible estimator known to dominate the James-Stein estimator is the generalized Bayes estimator relative to the fundamental harmonic function in three and higher dimension. The prior was suggested by Stein and the domination result is due to Kubokawa. Shao and Strawderman gave a class of estimators dominating the James-Stein positive-part estimator but were unable to demonstrate admissiblity of any in their class. Maruyama, following a suggestion of Stein, has studied generalized Bayes estimators which are members of a point mass at zero and a prior similar to the harmonic prior. He finds a subclass which is minimax and admissible but is unable to show that any in his class with positive point mass at zero dominate the James-Stein estimator. The results in this paper show that a subclass of Maruyama's procedures including the class that Stein conjectured might contain members dominating the James-Stein estimator cannot dominate the James-Stein estimator. We also show that under reasonable conditions, the “constant” in shrinkage factor must approachp-2 for domination to hold.     

A subclass of bayes linear estimators that are minimax

In the linear regression model with ellipsoidal parameter constraints, the problem of estimating the unknown parameter vector is studied. A well-described subclass of Bayes linear estimators is proposed in the paper. It is shown that for each member of this subclass, a generalized quadratic risk function exists so that the estimator is minimax. Moreover, some of the proposed Bayes linear estimators are admissible with respect to all possible generalized quadratic risks. Also, a necessary and sufficient condition is given to ensure that the considered Bayes linear estimator improves the least squares estimator over the whole ellipsoid whatever generalized risk function is chosen.     

On the admissibility of an estimator of a normal mean vector under a linex loss function

For ap-variate normal mean with known variances, the model proposed by Zellner (1986,J. Amer. Statist. Assoc.,81, 446–451) is discussed in a slightly different framework. A generalized Bayes estimate is derived from a three-stage Bayes point of view under the asymmetric loss function, and the admissibility of such estimators is proved.     

Estimation of a non-negative location parameter with unknown scale

For a vast array of general spherically symmetric location-scale models with a residual vector, we consider estimating the (univariate) location parameter when it is lower bounded. We provide conditions for estimators to dominate the benchmark minimax MRE estimator, and thus be minimax under scale invariant loss. These minimax estimators include the generalized Bayes estimator with respect to the truncation of the common non-informative prior onto the restricted parameter space for normal models under general convex symmetric loss, as well as non-normal models under scale invariant \(L^p\) loss with \(p>0\) . We cover many other situations when the loss is asymmetric, and where other generalized Bayes estimators, obtained with different powers of the scale parameter in the prior measure, are proven to be minimax. We rely on various novel representations, sharp sign change analyses, as well as capitalize on Kubokawa’s integral expression for risk difference technique. Several properties such as robustness of the generalized Bayes estimators under various loss functions are obtained.     

Spline-based sieve maximum likelihood estimation in the partly linear model under monotonicity constraints

We study a spline-based likelihood method for the partly linear model with monotonicity constraints. We use monotone B-splines to approximate the monotone nonparametric function and apply the generalized Rosen algorithm to compute the estimators jointly. We show that the spline estimator of the nonparametric component achieves the possible optimal rate of convergence under the smooth assumption and that the estimator of the regression parameter is asymptotically normal and efficient. Moreover, a spline-based semiparametric likelihood ratio test is established to make inference of the regression parameter. Also an observed profile information method to consistently estimate the standard error of the spline estimator of the regression parameter is proposed. A simulation study is conducted to evaluate the finite sample performance of the proposed method. The method is illustrated by an air pollution study.     

The 3/2th and 2nd order asymptotic efficiency of maximum probability estimators in non-regular cases

In this paper we consider the estimation problem on independent and identically distributed observations from a location parameter family generated by a density which is positive and symmetric on a finite interval, with a jump and a nonnegative right differential coefficient at the left endpoit. It is shown that the maximum probability estimator (MPE) is 3/2th order two-sided asymptotically efficient at a point in the sense that it has the most concentration probability around the true parameter at the point in the class of 3/2th order asymptotically median unbiased (AMU) estimators only when the right differential coefficient vanishes at the left endpoint. The second order upper bound for the concentration probability of second order AMU estimators is also given. Further, it is shown that the MPE is second order two-sided asymptotically efficient at a point in the above case only.Research supported by University of Tsukuba Project Research.     

Algorithme RLS en deux étapes pour l'estimation d'un modèle ARCH

The aim of this Note is, on the one hand, the development of a recursive method, in two stages, for estimating ARCH models, and, on the other hand, the analysis of the statistical properties of the estimators provided by this method. We show that these estimators are asymptotically Gaussian and that the estimator of the second stage is asymptotically efficient. To cite this article: A. Aknouche, H. Guerbyenne, C. R. Acad. Sci. Paris, Ser. I 343 (2006).     

Asymptotics of Huber-Dutter Estimators for Partial Linear Model with Nonstochastic Designs

For partial linear model Y=X~τβ_0 _(g0)(T) εwith unknown β_0∈R~d and an unknown smooth function go, this paper considers the Huber-Dutter estimators of β_0, scale σfor the errors and the function go respectively, in which the smoothing B-spline function is used. Under some regular conditions, it is shown that the Huber-Dutter estimators of β_0 and σare asymptotically normal with convergence rate n~((-1)/2) and the B-spline Huber-Dutter estimator of go achieves the optimal convergence rate in nonparametric regression. A simulation study demonstrates that the Huber-Dutter estimator of β_0 is competitive with its M-estimator without scale parameter and the ordinary least square estimator. An example is presented after the simulation study.