Mean integrated squared error of kernel estimators when the density and its derivative are not necessarily continuous

Summary Asymptotic properties of the mean integrated squared error (MISE) of kernel estimators of a density function, based on a sampleX ₁, …,X _n, were obtained by Rosenblatt [4] and Epanechnikov [1] for the case when the densityf and its derivativef′ are continuous. They found, under certain additional regularity conditions, that the optimal choiceh _n0 for the scale factorh _n=Kn^−α is given byh _n0=K₀n^−1/5 withK ₀ depending onf and the kernel; they also showed that MISE(h _n0)=O(n^−4/5) and Epanechnikov [1] found the optimal kernel. In this paper we investigate the robustness of these results to departures from the assumptions concerning the smoothness of the density function. In particular it is shown, under certain regularity conditions, that whenf is continuous but its derivativef′ is not, the optimal value of α in the scale factor becomes 1/4 and MISE(h _n0)=O(n^−3/4); for the case whenf is not continuous the optimal value of α becomes 1/2 and MISE(h _n0)=O(n^−1/2). For this last case the optimal kernel is shown to be the double exponential density. Supported by the Natural Sciences and Engineering Research Council of Canada under Grant Nr. A 3114 and by the Gouvernement du Québec, Programme de formation de chercheurs et d'action concertée.     

On the modal resolution of kernel density estimators

The ability of a kernel density estimator to resolve modes of the underlying density is investigated. For various bimodal densities and three different kernels, the smallest sample size required for the expectation of an optimally smoothed kernel estimator to be bimodal is determined. The optimality criterion employed is equivalent to asymptotic mean integrated squared error for sufficiently smooth densities.     

Fourier methods for smooth distribution function estimation

The limit behavior of the optimal bandwidth sequence for the kernel distribution function estimator is analyzed, in its greatest generality, by using Fourier transform methods. We show a class of distributions for which the kernel estimator achieves a first-order improvement in efficiency over the empirical estimator.     

Maximum integrated likelihood estimator of the interest parameter when the nuisance parameter is location or scale

The problem of estimation of an interest parameter in the presence of a nuisance parameter, which is either location or scale, is studied. Two estimators are considered: the usual maximum likelihood estimator and the estimator based on maximization of the integrated likelihood function. The estimators are compared, asymptotically, with respect to the bias and with respect to the mean squared error. The examples are given.     

When is the inverse regression estimator MSE-superior to the standard regression estimator in multivariate controlled calibration situations?

We assume as model a standard multivariate regression of y on x, fitted to a controlled calibration sample and used to estimate unknown x′s from observed y-values. The standard weighted least squares estimator (‘classical’, regress y on x and ‘solve’ for x) and the biased inverse regression estimator (regress x on y) are compared with respect to mean squared error. The regions are derived where the inverse regression estimator yields the smaller MSE. For any particular component of x this region is likely to contain ‘most’ future values in usual practice. For simultaneous estimation this needs not be true, however.     

Using the bootstrap to estimate mean squared error and select smoothing parameter in nonparametric problems

We describe a bootstrap method for estimating mean squared error and smoothing parameter in nonparametric problems. The method involves using a resample of smaller size than the original sample. There are many applications, which are illustrated using the special cases of nonparametric density estimation, nonparametric regression, and tail parameter estimation.     

Minimax invariant estimator of a continuous distribution function

Consider the problems of the continuous invariant estimation of a distribution function with a wide class of loss functions. It has been conjectured for long that the best invariant estimator is minimax for all sample sizes n1. This conjecture is proved in this short note.Partially supported by National Science Foundation Grant DMS 9001194.     

Analysis of the mean squared derivative cost function

In this paper, we investigate the mean squared derivative cost functions that arise in various applications such as in motor control, biometrics and optimal transport theory. We provide qualitative properties, explicit analytical formulas and computational algorithms for the cost functions. We also perform numerical simulations to illustrate the analytical results. In addition, as a by‐product of our analysis, we obtain an explicit formula for the inverse of a Wronskian matrix that is of independent interest in linear algebra and differential equations theory. Copyright © 2017 John Wiley & Sons, Ltd.     

Empirical likelihood estimators for the error distribution in nonparametric regression models

The aim of this paper is to show that existing estimators for the error distribution in non-parametric regression models can be improved when additional information about the distribution is included by the empirical likelihood method. The weak convergence of the resulting new estimator to a Gaussian process is shown and the performance is investigated by comparison of asymptotic mean squared errors and by means of a simulation study.      

A comparative simulation study on the IFS distribution function estimator

In this paper, we do a comparative simulation study of the standard empirical distribution function estimator versus a new class of nonparametric estimators of a distribution function F, called the iterated function system (IFS) estimator. The target distribution function F is supposed to have compact support. The IFS estimator of a distribution function F is considered as the fixed point of a contractive operator T defined in terms of a vector of parameters p and a family of affine maps which can be both dependent on the sample (X₁,X₂,…,X_n). Given , the problem consists in finding a vector p such that the fixed point of T is “sufficiently near” to F. It turns out that this is a quadratic constrained optimization problem that we propose to solve by penalization techniques. Analytical results prove that IFS estimators for F are asymptotically equivalent to the empirical distribution function (EDF) estimator. We will study the relative efficiency of the IFS estimators with respect to the empirical distribution function for small samples via the Monte Carlo approach.For well-behaved distribution functions F and for a particular family of the so-called wavelet maps the IFS estimators can be dramatically better than the empirical distribution function in the presence of missing data, i.e. when it is only possible to observe data on subsets of the whole support of F.     

Optimizing the smoothed bootstrap

In this paper we develop the technique of a generalized rescaling in the smoothed bootstrap, extending Silverman and Young's idea of shrinking. Unlike most existing methods of smoothing, with a proper choice of the rescaling parameter the rescaled smoothed bootstrap method produces estimators that have the asymptotic minimum mean (integrated) squared error, asymptotically improving existing bootstrap methods, both smoothed and unsmoothed. In fact, the new method includes existing smoothed bootstrap methods as special cases. This unified approach is investigated in the problems of estimation of global and local functionals and kernel density estimation. The emphasis of this investigation is on theoretical improvements which in some cases offer practical potential.     

Optimal kernel estimation of densities

Precise asymptotic behavior for mean integrated squared error (MISE) is determined for sequences of kernel estimators of a density in a broad class, including discontinuous and possibly unbounded densities. The paper shows that the sequence using the kernel optimal at each fixed sample size is asymptotically more efficient than a sequence generated by changing the bandwidth of a fixed kernel shape, regardless of the kernel shape. The class of densities considered are those whose characteristic functions behave at large arguments like the product of a Fourier series and a regularly varying function. This condition may be related to the smoothness of an m-th derivative of the density.Partially supported by National Science Foundation Grant DMS-8711924.     

A Bootstrap-based Method to Achieve Optimality in Estimating the Extreme-value Index

Estimators of the extreme-value index are based on a set of upper order statistics. We present an adaptive method to choose the number of order statistics involved in an optimal way, balancing variance and bias components. Recently this has been achieved for the similar but some what less involved case of regularly varying tails (Drees and Kaufmann, 1997); Danielsson et al., 1996). The present paper follows the line of proof of the last mentioned paper.     

Histogram-kernel error and its application for bin width selection in histograms

Histogram and kernel estimators are usually regarded as the two main classical data-based non- parametric tools to estimate the underlying density functions for some given data sets. In this paper we will integrate them and define a histogram-kernel error based on the integrated square error between histogram and binned kernel density estimator, and then exploit its asymptotic properties. Just as indicated in this paper, the histogram-kernel error only depends on the choice of bin width and the data for the given prior kernel densities. The asymptotic optimal bin width is derived by minimizing the mean histogram-kernel error. By comparing with Scott’s optimal bin width formula for a histogram, a new method is proposed to construct the data-based histogram without knowledge of the underlying density function. Monte Carlo study is used to verify the usefulness of our method for different kinds of density functions and sample sizes.     

Limit distribution of the integrated squared error of trigonometric series regression estimator

Limit distribution is studied for the integrated squared error of the projection regression estimator (2) constructed on the basis of independent observations (1). By means of the obtained limit theorems, a test is given for verifying the hypothesis on the regression, and the power of this test is calculated in the case of Pitman alternatives.     

Precise asymptotics of error variance estimator in partially linear models

In this paper, we focus our attention on the precise asymptotics of error variance estimator in partially linear regression models, y _i = x _i ^τ β + g(t _i ) + ε _i , 1 ≤ i ≤ n, {ε _i , i = 1, ⋯ n} are i.i.d random errors with mean 0 and positive finite variance σ ². Following the ideas of Allan Gut and Aurel Spătaru^[7,8] and Zhang^[21], on precise asymptotics in the Baum-Katz and Davis laws of large numbers and precise rate in laws of the iterated logarithm, respectively, and subject to some regular conditions, we obtain the corresponding results in partially linear regression models.      

An efficient estimator for the expectation of a bounded function under the residual distribution of an autoregressive process

Consider a stationary first-order autoregressive process, with i.i.d. residuals following an unknown mean zero distribution. The customary estimator for the expectation of a bounded function under the residual distribution is the empirical estimator based on the estimated residuals. We show that this estimator is not efficient, and construct a simple efficient estimator. It is adaptive with respect to the autoregression parameter.     

Deconvolution from panel data with unknown error distribution

We devise a new method of estimating a distribution in a deconvolution model with panel data and an unknown distribution of the additive errors. We prove strong consistency under a minimal condition concerning the zero sets of the involved characteristic functions.     

The Bahadur representation for kernel-type estimator of the quantile function under strong mixing and censored data

In this paper, we consider the kernel-type estimator of the quantile function based on the kernel smoother under a censored dependent model. The Bahadur-type representation of the kernel smooth estimator is established, and from the Bahadur representation we can show that this estimator is strongly consistent.