Efficient Density Estimation for Ergodic Diffusion Processes

The problem of asymptotically efficient estimation of the density of invariant measure of a diffusion process is considered. The efficient estimator is defined with the help of the minimax lower bound on the risk of all estimators. We show that the local–time and kernel–type estimators are asymptotically efficient for the loss functions with polynomial majorants. The asymptotic behavior of a wide class of unbiased estimators with the same limit variances is also discussed. This revised version was published online in June 2006 with corrections to the Cover Date.     

Local asymptotic normality and asymptotical minimax efficiency of the MLE under random censorship

Here we study the problems of local asymptotic normality of the parametric family of distributions and asymptotic minimax efficient estimators when the observations are subject to right censoring. Local asymptotic normality will be established under some mild regularity conditions. A lower bound for local asymptotic minimax risk is given with respect to a bowl-shaped loss function, and furthermore a necessary and sufficient condition is given in order to achieve this lower bound. Finally, we show that this lower bound can be attained by the maximum likelihood estimator in the censored case and hence it is local asymptotic minimax efficient.     

The Minimax Estimator of Stochastic Regression Coefficients and Parameters in the Class of All Estimators

In this paper, the authors address the problem of the minimax estimator of linear combinations of stochastic regression coefficients and parameters in the general normal linear model with random effects. Under a quadratic loss function, the minimax property of linear estimators is investigated. In the class of all estimators, the minimax estimator of estimable functions, which is unique with probability 1, is obtained under a multivariate normal distribution.     

Semiparametric Estimation of the State of a Dynamical System with Small Noise

We consider the problem of estimation of the state of a perturbed dynamical system by observing the trajectory of a diffusion process with known small diffusion coefficient. The drift coefficient is supposed to be an unknown regular function. Asymptotic minimax lower bounds for the risk of any estimator of the state are derived and the notion of efficiency is introduced. The naïve estimator for this problem is proposed and its basic properties are discussed. It emerges that this estimator is asymptotically efficient.     

Multivariate Locally Weighted Polynomial Fitting and Partial Derivative Estimation

Nonparametric regression estimator based on locally weighted least squares fitting has been studied by Fan and Ruppert and Wand. The latter paper also studies, in the univariate case, nonparametric derivative estimators given by a locally weighted polynomial fitting. Compared with traditional kernel estimators, these estimators are often of simpler form and possess some better properties. In this paper, we develop current work on locally weighted regression and generalize locally weighted polynomial fitting to the estimation of partial derivatives in a multivariate regression context. Specifically, for both the regression and partial derivative estimators we prove joint asymptotic normality and derive explicit asymptotic expansions for their conditional bias and conditional convariance matrix (given observations of predictor variables) in each of the two important cases of local linear fit and local quadratic fit.     

Classical Backfitting for Smooth-Backfitting Additive Models

Smooth backfitting has been shown to have better theoretical properties than classical backfitting for fitting additive models based on local linear regression. In this article, we show that the smooth backfitting procedure in the local linear case can be alternatively performed as a classical backfitting procedure with a different type of smoother matrices. These smoother matrices are symmetric and shrinking and some established results in the literature are readily applicable. The connections allow the smooth backfitting algorithm to be implemented in a much simplified way, give new insights on the differences between the two approaches in the literature, and provide an extension to local polynomial regression. The connections also give rise to a new estimator at data points. Asymptotic properties of general local polynomial smooth backfitting estimates are investigated, allowing for different orders of local polynomials and different bandwidths. Cases of oracle efficiency are discussed. Computer-generated simulations are conducted to demonstrate finite sample behaviors of the methodology and a real data example is given for illustration. Supplementary materials for this article are available online.     

Minimax estimation and the least squares method

This paper is devoted to the problem of minimax estimation of parameters in linear regression models with uncertain second order statistics. The solution to the problem is shown to be the least squares estimator corresponding to the least favourable matrix of the second moments. This allows us to construct a new algorithm for minimax estimation closely connected with the least squares method. As an example, we consider the problem of polynomial regression introduced by A. N. Kolmogorov     

The FEXP estimator for potentially non-stationary linear time series

We consider semiparametric fractional exponential (FEXP) estimators of the memory parameter d for a potentially non-stationary linear long-memory time series with additive polynomial trend. We use differencing to annihilate the polynomial trend, followed by tapering to handle the potential non-invertibility of the differenced series. We propose a method of pooling the tapered periodogram which leads to more efficient estimators of d than existing pooled, tapered estimators. We establish asymptotic normality of the tapered FEXP estimator in the Gaussian case with or without pooling. We establish asymptotic normality of the estimator in the linear case if pooling is used. Finally, we consider minimax rate-optimality and feasible nearly rate-optimal estimators in the Gaussian case.     

Minimax efficient finite-difference stochastic gradient estimators using black-box function evaluations

Standard approaches to stochastic gradient estimation, with only noisy black-box function evaluations, use the finite-difference method or its variants. Though natural, it is open to our knowledge whether their statistical accuracy is the best possible. This paper argues so by showing that central finite-difference is a nearly minimax optimal zeroth-order gradient estimator for a suitable class of objective functions and mean squared risk, among both the class of linear estimators and the much larger class of all (nonlinear) estimators.     

Optimal designs with respect to Elfving's partial minimax criterion in polynomial regression

For the polynomial regression model on the interval [a, b] the optimal design problem with respect to Elfving's minimax criterion is considered. It is shown that the minimax problem is related to the problem of determining optimal designs for the estimation of the individual parameters. Sufficient conditions are given guaranteeing that an optimal design for an individual parameter in the polynomial regression is also minimax optimal for a subset of the parameters. The results are applied to polynomial regression on symmetric intervals [–b, b] (b1) and on nonnegative or nonpositive intervals where the conditions reduce to very simple inequalities, involving the degree of the underlying regression and the index of the maximum of the absolute coefficients of the Chebyshev polynomial of the first kind on the given interval. In the most cases the minimax optimal design can be found explicitly.Research supported in part by the Deutsche Forschungsgemeinschaft.Research supported in part by NSF Grant DMS 9101730.     

Asymptotically Local Minimax Estimation of Infinitely Smooth Density with Censored Data

The problem of the nonparametric minimax estimation of an infinitely smooth density at a given point, under random censorship, is considered. We establish the exact asymptotics of the local minimax risk and propose the efficient kernel-type estimator based on the well known Kaplan-Meier estimator.     

Stein shrinkage and second-order efficiency for semiparametric estimation of the shift

The problem of estimating the shift (or, equivalently, the center of symmetry) of an unknown symmetric and periodic function f observed in Gaussian white noise is considered. Using the blockwise Stein method, a penalized profile likelihood with a data-driven penalization is introduced so that the estimator of the center of symmetry is defined as the maximizer of the penalized profile likelihood. This estimator has the advantage of being independent of the functional class to which the signal f is assumed to belong and, furthermore, is shown to be semiparametrically adaptive and efficient. Moreover, the second-order term of the risk expansion of the proposed estimator is proved to behave at least as well as the second-order term of the risk of the best possible estimator using monotone smoothing filter. Under mild assumptions, this estimator is shown to be second-order minimax sharp adaptive over the whole scale of Sobolev balls with smoothness β > 1. Thus, these results extend those of [10], where second-order asymptotic minimaxity is proved for an estimator depending on the functional class containing f and β ≥ 2 is required.      

On Bayes and unbiased estimators of loss

We consider estimation of loss for generalized Bayes or pseudo-Bayes estimators of a multivariate normal mean vector, θ. In 3 and higher dimensions, the MLEX is UMVUE and minimax but is inadmissible. It is dominated by the James-Stein estimator and by many others. Johnstone (1988, On inadmissibility of some unbiased estimates of loss,Statistical Decision Theory and Related Topics, IV (eds. S. S. Gupta and J. O. Berger), Vol. 1, 361–379, Springer, New York) considered the estimation of loss for the usual estimatorX and the James-Stein estimator. He found improvements over the Stein unbiased estimator of risk. In this paper, for a generalized Bayes point estimator of θ, we compare generalized Bayes estimators to unbiased estimators of loss. We find, somewhat surprisingly, that the unbiased estimator often dominates the corresponding generalized Bayes estimator of loss for priors which give minimax estimators in the original point estimation problem. In particular, we give a class of priors for which the generalized Bayes estimator of θ is admissible and minimax but for which the unbiased estimator of loss dominates the generalized Bayes estimator of loss. We also give a general inadmissibility result for a generalized Bayes estimator of loss. Research supported by NSF Grant DMS-97-04524.     

Frontier estimation with local polynomials and high power-transformed data

We present a new method for estimating the frontier of a sample. The estimator is based on a local polynomial regression on the power-transformed data. We assume that the exponent of the transformation goes to infinity while the bandwidth goes to zero. We give conditions on these two parameters for obtaining almost complete convergence. The asymptotic conditional bias and variance of the estimator are provided and its good performance is illustrated for some finite sample situations.     

Nonparametric regression estimation with general parametric error covariance

The asymptotic distribution for the local linear estimator in nonparametric regression models is established under a general parametric error covariance with dependent and heterogeneously distributed regressors. A two-step estimation procedure that incorporates the parametric information in the error covariance matrix is proposed. Sufficient conditions for its asymptotic normality are given and its efficiency relative to the local linear estimator is established. We give examples of how our results are useful in some recently studied regression models. A Monte Carlo study confirms the asymptotic theory predictions and compares our estimator with some recently proposed alternative estimation procedures.     

Asymptotic minimax properties of M-estimators of scale

We ask whether or not the saddlepoint property holds, for robust M-estimation of scale, in gross-errors and Kolmogorov neighbourhoods of certain distributions. This is of interest since the saddlepoint property implies the minimax property — that the supremum of the asymptotic variance of an M-estimator is minimized by the maximum likelihood estimator for that member of the distributional class with minimum Fisher information. Our findings are exclusively negative — the saddlepoint property fails in all cases investigated.     

Weyl eigenvalue asymptotics and sharp adaptation on vector bundles

This paper examines the estimation of an indirect signal embedded in white noise on vector bundles. It is found that the sharp asymptotic minimax bound is determined by the degree to which the indirect signal is embedded in the linear operator. Thus when the linear operator has polynomial decay, recovery of the signal is polynomial where the exact minimax constant and rate are determined. Adaptive sharp estimation is carried out using a blockwise shrinkage estimator. Application to the spherical deconvolution problem for the polynomially bounded case is made.     

Adaptive estimation of linear functionals in functional linear models

We consider the estimation of the value of a linear functional of the slope parameter in functional linear regression, where scalar responses are modeled in dependence of randomfunctions. In Johannes and Schenk [2010] it has been shown that a plug-in estimator based on dimension reduction and additional thresholding can attain minimax optimal rates of convergence up to a constant. However, this estimation procedure requires an optimal choice of a tuning parameter with regard to certain characteristics of the slope function and the covariance operator associated with the functional regressor. As these are unknown in practice, we investigate a fully data-driven choice of the tuning parameter based on a combination of model selection and Lepski??s method, which is inspired by the recent work of Goldenshluger and Lepski [2011]. The tuning parameter is selected as theminimizer of a stochastic penalized contrast function imitating Lepski??smethod among a random collection of admissible values. We show that this adaptive procedure attains the lower bound for the minimax risk up to a logarithmic factor over a wide range of classes of slope functions and covariance operators. In particular, our theory covers pointwise estimation as well as the estimation of local averages of the slope parameter.     

Minimax and minimax adaptive estimation in multiplicative regression: locally bayesian approach

This paper deals with the non-parametric estimation in the regression with the multiplicative noise. Using the local polynomial fitting and the bayesian approach, we construct the minimax on isotropic H?lder class estimator. Next, applying Lepski’s method we propose an estimator which is optimally adaptive over the collection of isotropic H?lder classes. To prove the optimality of the proposed procedure, we establish in particular the exponential inequality for the deviation of locally bayesian estimator since the parameter estimated. These theoretical results are illustrated by simulation study.     

Optimal and superoptimal convergence rate of the local linear estimator of nonparametric regression function in continuous time

We consider the estimation problem of the nonparametric regression in continuous time by the local linear estimator in the asymptotic quadratic error sense. In suitable conditions of strongly mixing and that of irregularity, we obtained optimal and superoptimal convergence rate of the estimator.