Mean integrated squared error of kernel estimators when the density and its derivative are not necessarily continuous

Summary Asymptotic properties of the mean integrated squared error (MISE) of kernel estimators of a density function, based on a sampleX ₁, …,X _n, were obtained by Rosenblatt [4] and Epanechnikov [1] for the case when the densityf and its derivativef′ are continuous. They found, under certain additional regularity conditions, that the optimal choiceh _n0 for the scale factorh _n=Kn^−α is given byh _n0=K₀n^−1/5 withK ₀ depending onf and the kernel; they also showed that MISE(h _n0)=O(n^−4/5) and Epanechnikov [1] found the optimal kernel. In this paper we investigate the robustness of these results to departures from the assumptions concerning the smoothness of the density function. In particular it is shown, under certain regularity conditions, that whenf is continuous but its derivativef′ is not, the optimal value of α in the scale factor becomes 1/4 and MISE(h _n0)=O(n^−3/4); for the case whenf is not continuous the optimal value of α becomes 1/2 and MISE(h _n0)=O(n^−1/2). For this last case the optimal kernel is shown to be the double exponential density. Supported by the Natural Sciences and Engineering Research Council of Canada under Grant Nr. A 3114 and by the Gouvernement du Québec, Programme de formation de chercheurs et d'action concertée.