Strong consistency of density estimation by orthogonal series methods for dependent variables with applications

Among several widely use methods of nonparametric density estimation is the technique of orthogonal series advocated by several authors. For such estimate when the observations are assumed to have been taken from strong mixing sequence in the sense of Rosenblatt [7] we study strong consistency by developing probability inequality for bounded strongly mixing random variables. The results obtained are then applied to two estimates of the functional Δ(f)=∫f ² (x)dx were strong consistency is established. One of the suggested two estimates of Δ(f) was recently studied by Schuler and Wolff [8] in the case of independent and identically distributed observations where they established consistency in the second mean of the estimate. Research supported in part by the National Research Council of Canada and in part by McMaster University Research Board. Now at Memphis State University, Memphis, Tennessee 38152, U.S.A.     

Speed of convergence in nonparametric kernel estimation of a regression function and its derivatives

Summary The objective in nonparametric regression is to infer a functiong(x) and itspth order derivativesg ^(g)(x),p≧1 fixed, on the basis of a finite collection of pairs {x _i, g(x_i)+Z _i} _i=1 ⁿ , where the noise componentsZ _i satisfy certain modest assumptions and the domain pointsx _i are selected non-randomly. This paper exhibits a new class of kernel estimatesg _n ^(p) ,p≧0 fixed. The main theoretical results of this study are the rates of convergence obtained for mean square and strong consistency ofg _n ^(p) each of them being uniform on the (0,1).     

Efficient nonparametric estimation of distribution density in the basis of algebraic polynomials

A nonparametric estimatef ^* of an unknown distribution densityf W is called locally minimax iff it is minimax for all not too small neighborhoodsW _g ,g W, simultaneously, whereW is some dense subset ofW. Radaviius and Rudzkis proved the existence of such an estimate under some general conditions. However, the construction of the estimate is rather complicated. In this paper, a new estimate is proposed. This estimate is locally minimax under some additional assumptions which usually hold for orthobases of algebraic polynomial and is almost as simple as the linear projective estimate. Thus, it takes a form convenient for the construction of an adaptive estimator, which does not usea-priori information about the smoothness of the density. The adaptive estimation problem is briefly discussed and an unknown density fitting by Jacobi polynomials is investigated more explicitly.     

Integrated mean square properties of density estimation by orthogonal series methods for dependent variables

Summary The rates at which integrated mean square and mean squre errors of nonparametric density estimation by orthogonal series method for sequences of strictly stationary strong mixing random variables are obtained. These rates are better than those known to hold for the independent case and they are shown to hold for Markov processes. In fact our results when specialized to the independent case are improvements over previously known results of Schwartz (1967,Ann. Math. Statist.,38, 1262–1265). An extension of the results to estimation of the bivariate density is also given. Research supported by a faculty summer research grant MS-STAT-42 from the University of Petroleum and Minerals.     

Nonparametric estimation of the location and scale parameters based on density estimation

Summary By representing the location and scale parameters of an absolutely continuous distribution as functionals of the usually unknown probability density function, it is possible to provide estimates of these parameters in terms of estimates of the unknown functionals. Using the properties of well-known methods of density estimates, it is shown that the proposed estimates possess nice large sample properties and it is indicated that they are also robust against dependence in the sample. The estimates perform well against other estimates of location and scale parameters.     

Some inequalities for strong mixing random variables with applications to density estimation

In this paper, we establish an inequality of the characteristic functions for strongly mixing random vectors, by which, an upper bound is provided for the supremum of the absolute value of the difference of two multivariate probability density functions based on strongly mixing random vectors. As its application, we consider the consistency and asymptotic normality of a kernel estimate of a density function under strong mixing. Our results generalize some known results in the literature.     

A moving window approach for nonparametric estimation of the conditional tail index

We present a nonparametric family of estimators for the tail index of a Pareto-type distribution when covariate information is available. Our estimators are based on a weighted sum of the log-spacings between some selected observations. This selection is achieved through a moving window approach on the covariate domain and a random threshold on the variable of interest. Asymptotic normality is proved under mild regularity conditions and illustrated for some weight functions. Finite sample performances are presented on a real data study.     

On estimation of a density and its derivatives

Summary Letf _n ^(p) be a recursive kernel estimate off ^(p) thepth order derivative of the probability density functionf, based on a random sample of sizen. In this paper, we provide bounds for the moments of and show that the rate of almost sure convergence of to zero isO(n ^−α), α<(r−p)/(2r+1), iff ^(r),r>p≧0, is a continuousL ₂(−∞, ∞) function. Similar rate-factor is also obtained for the almost sure convergence of to zero under different conditions onf. This work was supported in part by the Research Foundation of SUNY.     

Nonlinear weighted least squares estimation of a three-parameter Weibull density with a nonparametric start

This paper is concerned with the parameter estimation problem for the three-parameter Weibull density which is widely employed as a model in reliability and lifetime studies. Our approach is a combination of nonparametric and parametric methods. The basic idea is to start with an initial nonparametric density estimate which needs to be as good as possible, and then apply the nonlinear least squares method to estimate the unknown parameters. As a main result, a theorem on the existence of the least squares estimate is obtained. Some simulations are given to show that our approach is satisfactory if the initial density is of good enough quality.     

Functional nonparametric estimation of conditional extreme quantiles

We address the estimation of quantiles from heavy-tailed distributions when functional covariate information is available and in the case where the order of the quantile converges to one as the sample size increases. Such “extreme” quantiles can be located in the range of the data or near and even beyond the boundary of the sample, depending on the convergence rate of their order to one. Nonparametric estimators of these functional extreme quantiles are introduced, their asymptotic distributions are established and their finite sample behavior is investigated.     

Asymptotic properties of nonparametric curve estimates

We consider a class of nonparametric estimators for the regression functionm(t) in the model:y _i =m(t _i ) + _i , 1 i n, t _i [0, 1], which are linear in the observationsy _i . Several limit theorems concerning local and global stochastic and a.s. convergence and limit distributions are given.     

Multi-stage nonparametric estimation of density function using orthonormal systems

In this note, the authors propose a new nonparametric method of estimation of density using orthonormal systems iteratively. The asymptotic mean integrated square error of the estimate at each stage is less than or equal to that of the preceding stage. The new estimate is better, in some cases, than the traditional estimate based upon orthonormal functions from the point of view of the mean integrated square error in the limit.     

Estimating a density on the positive half line by the method of orthogonal series

The kernel method of density estimation is not so attractive when the density has its support confined to (0, ∞), particularly when the density is unsmooth at the origin. In this situation the method of orthogonal series is competitive. We consider three essentially different orthogonal series—those based on the even and odd Hermite functions, respectively, and that based on Laguerre functions—and compare them from the point of view of mean integrated square error.     

Strong consistency properties of nonparametric estimators for randomly censored data,II: Estimation of density and failure rate

This article is Part II of a two-part study. Properties of the product-limit estimator established in the previous part [2] are now used to prove the strong consistency of some nonparametric density and failure rate estimators which can be used with randomly censored data.The third author's research was partly supported by the National Research Council of Canada.     

L p-consistency of multivariate density estimates

L _p notion of the weak, mean, and strong consistency of the kernel method of multivariate density estimation is proposed and studied. The results expand, unify, or generalize most known results in the literature. Rates of convergence in mean and strongL _p-consistencies are presented.     

Residual variance estimation using a nearest neighbor statistic

In this paper we consider the problem of estimating E[(Y−E[Y∣X])²] based on a finite sample of independent, but not necessarily identically distributed, random variables . We analyze the theoretical properties of a recently developed estimator. It is shown that the estimator has many theoretically interesting properties, while the practical implementation is simple.     

A note on weak convergence of mean residual life of stationary mixing random variables

For a sequence of strictly stationary uniform or strong mixing we estimate the mean residual time of the marginal distribution from the first n observations. Under appropriate conditions it is shown that the estimate converges weakly to a well-defined Gaussian process even when the sample size is random.     

Adaptive estimation of the transition density of a particular hidden Markov chain

We study the following model of hidden Markov chain: with (X_i) a real-valued positive recurrent and stationary Markov chain, and (?_i)_1?i?n+1 a noise independent of the sequence (X_i) having a known distribution. We present an adaptive estimator of the transition density based on the quotient of a deconvolution estimator of the density of X_i and an estimator of the density of (X_i,X_i+1). These estimators are obtained by contrast minimization and model selection. We evaluate the L² risk and its rate of convergence for ordinary smooth and supersmooth noise with regard to ordinary smooth and supersmooth chains. Some examples are also detailed.     

A Berry-Esseen type bound in kernel density estimation for strong mixing censored samples

In this paper, we discuss the estimation of a density function based on censored data by the kernel smoothing method when the survival and the censoring times form a stationary α-mixing sequence. A Berry-Esseen type bound is derived for the kernel density estimator at a fixed point x. For practical purposes, a randomly weighted estimator of the density function is also constructed and investigated.     

Estimation of density functions by order statistics

The properties of the empirical density function,f _n(x) = k/n( _j ⁺ – _j-1 ⁺ ) if _j-1 ⁺ < x ⁺ where _j-1 ⁺ and _j ⁺ are sample elements and there are exactlyk – 1 sample elements between them, are studied in that practical point of view how to choose a suitablek for a good estimation. A bound is given for the expected value of the absolute value of difference between the empirical and theoretical density functions.