On near neighbour estimates of a multivariate density

A recent paper by Mack and Rosenblatt (J. Multivar. Anal.9 (1979), 1–15) has shown that near neighbour estimators of a density may perform more poorly than other kernel-type estimators, particularly for x values in the tail of a distribution. In order to overcome the difficulties discovered by Mack and Rosenblatt, a generalized type of near neighbour estimator is proposed. Here the window size, or bandwidth, is chosen as a function of near neighbour distances, rather than actually equal to one of the distances. Two forms for this function are suggested and it is proved that for large samples the resulting estimator does not suffer the drawbacks of the usual near neighbour estimator.     

Asymptotic properties of nonparametric curve estimates

We consider a class of nonparametric estimators for the regression functionm(t) in the model:y _i =m(t _i ) + _i , 1 i n, t _i [0, 1], which are linear in the observationsy _i . Several limit theorems concerning local and global stochastic and a.s. convergence and limit distributions are given.     

Estimation of density functions by order statistics

The properties of the empirical density function,f _n(x) = k/n( _j ⁺ – _j-1 ⁺ ) if _j-1 ⁺ < x ⁺ where _j-1 ⁺ and _j ⁺ are sample elements and there are exactlyk – 1 sample elements between them, are studied in that practical point of view how to choose a suitablek for a good estimation. A bound is given for the expected value of the absolute value of difference between the empirical and theoretical density functions.     

A note on nonparametric density estimation for dependent variables using a delta sequence

Summary A general method based on “delta sequences” due to Walter and Blum [12] is extended to sequences of strictly stationary mixing random variables having the same marginal distribution admitting a Lebesgue probability density function. It is proved that, under certain conditions, the rate of mean square convergence obtained in the i.i.d. case by Walter and Blum, continues to hold. University of Petroleum and Minerals     

Estimating a density on the positive half line by the method of orthogonal series

The kernel method of density estimation is not so attractive when the density has its support confined to (0, ∞), particularly when the density is unsmooth at the origin. In this situation the method of orthogonal series is competitive. We consider three essentially different orthogonal series—those based on the even and odd Hermite functions, respectively, and that based on Laguerre functions—and compare them from the point of view of mean integrated square error.     

Tree-based multivariate regression and density estimation with right-censored data

We propose a unified strategy for estimator construction, selection, and performance assessment in the presence of censoring. This approach is entirely driven by the choice of a loss function for the full (uncensored) data structure and can be stated in terms of the following three main steps. (1) First, define the parameter of interest as the minimizer of the expected loss, or risk, for a full data loss function chosen to represent the desired measure of performance. Map the full data loss function into an observed (censored) data loss function having the same expected value and leading to an efficient estimator of this risk. (2) Next, construct candidate estimators based on the loss function for the observed data. (3) Then, apply cross-validation to estimate risk based on the observed data loss function and to select an optimal estimator among the candidates. A number of common estimation procedures follow this approach in the full data situation, but depart from it when faced with the obstacle of evaluating the loss function for censored observations. Here, we argue that one can, and should, also adhere to this estimation road map in censored data situations.Tree-based methods, where the candidate estimators in Step 2 are generated by recursive binary partitioning of a suitably defined covariate space, provide a striking example of the chasm between estimation procedures for full data and censored data (e.g., regression trees as in CART for uncensored data and adaptations to censored data). Common approaches for regression trees bypass the risk estimation problem for censored outcomes by altering the node splitting and tree pruning criteria in manners that are specific to right-censored data. This article describes an application of our unified methodology to tree-based estimation with censored data. The approach encompasses univariate outcome prediction, multivariate outcome prediction, and density estimation, simply by defining a suitable loss function for each of these problems. The proposed method for tree-based estimation with censoring is evaluated using a simulation study and the analysis of CGH copy number and survival data from breast cancer patients.     

Asymptotic normality of a quadratic measure of orthogonal series type density estimate

LetX ₁,...,X _n be i.i.d. random variable with a common densityf. Let be an estimate off(x) based on a complete orthonormal basis {φ _k :k≧0} ofL ₂[a, b]. A Martingale central limit theorem is used to show that , where and .     

Nonparametric estimation of the location and scale parameters based on density estimation

Summary By representing the location and scale parameters of an absolutely continuous distribution as functionals of the usually unknown probability density function, it is possible to provide estimates of these parameters in terms of estimates of the unknown functionals. Using the properties of well-known methods of density estimates, it is shown that the proposed estimates possess nice large sample properties and it is indicated that they are also robust against dependence in the sample. The estimates perform well against other estimates of location and scale parameters.     

Nonparametric estimation of mixed partial derivatives of a multivariate density

On the basis of a random sample of size n on an m-dimensional random vector X, this note proposes a class of estimators f_n^(p) of f^(p), where f is a density of X w.r.t. a σ-finite measure dominated by the Lebesgue measure on R^m, p = (p₁,…,p_m), p_j ≥ 0, fixed integers, and for x = (x₁,…,x_m) in R^m, f^(p)(x) = ?^p₁+…+p_m f(x)/(?^p₁x₁ … ?^p_mx_m). Asymptotic unbiasedness as well as both almost sure and mean square consistencies of f_n^(p) are examined. Further, a necessary and sufficient condition for uniform asymptotic unbisedness or for uniform mean square consistency of f_n^(p) is given. Finally, applications of estimators of this note to certain statistical problems are pointed out.     

Symmetrised M-estimators of multivariate scatter

In this paper we introduce a family of symmetrised M-estimators of multivariate scatter. These are defined to be M-estimators only computed on pairwise differences of the observed multivariate data. Symmetrised Huber's M-estimator and Dümbgen's estimator serve as our examples. The influence functions of the symmetrised M-functionals are derived and the limiting distributions of the estimators are discussed in the multivariate elliptical case to consider the robustness and efficiency properties of estimators. The symmetrised M-estimators have the important independence property; they can therefore be used to find the independent components in the independent component analysis (ICA).     

Simple and efficient improvements of multivariate local linear regression

This paper studies improvements of multivariate local linear regression. Two intuitively appealing variance reduction techniques are proposed. They both yield estimators that retain the same asymptotic conditional bias as the multivariate local linear estimator and have smaller asymptotic conditional variances. The estimators are further examined in aspects of bandwidth selection, asymptotic relative efficiency and implementation. Their asymptotic relative efficiencies with respect to the multivariate local linear estimator are very attractive and increase exponentially as the number of covariates increases. Data-driven bandwidth selection procedures for the new estimators are straightforward given those for local linear regression. Since the proposed estimators each has a simple form, implementation is easy and requires much less or about the same amount of effort. In addition, boundary corrections are automatic as in the usual multivariate local linear regression.     

Rank approach to the multivariate two-population mixture problem

Given three independent multivariate samples, of which, two are from unknown populations that are known to be distinct and the other is from an unknown mixture of the two, the problem of estimation of the mixture rate is considered. A procedure based on linearly compounded rank-scores is studied and the problem of optimisation with respect to the compounding coefficients so as to minimize the asymptotic variance of the estimate is solved.     

On estimation of a density and its derivatives

Summary Letf _n ^(p) be a recursive kernel estimate off ^(p) thepth order derivative of the probability density functionf, based on a random sample of sizen. In this paper, we provide bounds for the moments of and show that the rate of almost sure convergence of to zero isO(n ^−α), α<(r−p)/(2r+1), iff ^(r),r>p≧0, is a continuousL ₂(−∞, ∞) function. Similar rate-factor is also obtained for the almost sure convergence of to zero under different conditions onf. This work was supported in part by the Research Foundation of SUNY.     

Asymptotic properties for estimates of nonparametric regression models based on negatively associated sequences

Consider the nonparametric regression model Y_ni=g(x_ni)+ε_ni for i=1,…,n, where g is unknown, x_ni are fixed design points, and ε_ni are negatively associated random errors. Nonparametric estimator g_n(x) of g(x) will be introduced and its asymptotic properties are studied. In particular, the pointwise and uniform convergence of g_n(x) and its asymptotic normality will be investigated. This extends the earlier work on independent random errors (e.g. see J. Multivariate Anal. 25(1) (1988) 100).     

Speed of convergence in nonparametric kernel estimation of a regression function and its derivatives

Summary The objective in nonparametric regression is to infer a functiong(x) and itspth order derivativesg ^(g)(x),p≧1 fixed, on the basis of a finite collection of pairs {x _i, g(x_i)+Z _i} _i=1 ⁿ , where the noise componentsZ _i satisfy certain modest assumptions and the domain pointsx _i are selected non-randomly. This paper exhibits a new class of kernel estimatesg _n ^(p) ,p≧0 fixed. The main theoretical results of this study are the rates of convergence obtained for mean square and strong consistency ofg _n ^(p) each of them being uniform on the (0,1).     

Multivariate k-nearest neighbor density estimates

Under appropriate assumptions, expressions describing the asymptotic behavior of the bias and variance of k-nearest neighbor density estimates with weight function w are obtained. The behavior of these estimates is compared with that of kernel estimates. Particular attention is paid to the properties of the estimates in the tail.     

On the risk of estimates for block decreasing densities

A density f=f(x₁,…,x_d) on [0,∞)^d is block decreasing if for each j∈{1,…,d}, it is a decreasing function of x_j, when all other components are held fixed. Let us consider the class of all block decreasing densities on [0,1]^d bounded by B. We shall study the minimax risk over this class using n i.i.d. observations, the loss being measured by the L₁ distance between the estimate and the true density. We prove that if S=log(1+B), lower bounds for the risk are of the form C(S^d/n)^1/(d+2), where C is a function of d only. We also prove that a suitable histogram with unequal bin widths as well as a variable kernel estimate achieve the optimal multivariate rate. We present a procedure for choosing all parameters in the kernel estimate automatically without loosing the minimax optimality, even if B and the support of f are unknown.     

Strong consistency properties of nonparametric estimators for randomly censored data,II: Estimation of density and failure rate

This article is Part II of a two-part study. Properties of the product-limit estimator established in the previous part [2] are now used to prove the strong consistency of some nonparametric density and failure rate estimators which can be used with randomly censored data.The third author's research was partly supported by the National Research Council of Canada.     

Nonparametric iterative estimation of multivariate binary density

In this paper, an iterative estimate of the multivariate density is proposed when the variables are binary in nature. Some properties of this estimate are also discussed. Finally, applications of this estimate are discussed in the areas of pattern recognition and reliability.