Efficient nonparametric estimation of distribution density in the basis of algebraic polynomials

A nonparametric estimatef ^* of an unknown distribution densityf W is called locally minimax iff it is minimax for all not too small neighborhoodsW _g ,g W, simultaneously, whereW is some dense subset ofW. Radaviius and Rudzkis proved the existence of such an estimate under some general conditions. However, the construction of the estimate is rather complicated. In this paper, a new estimate is proposed. This estimate is locally minimax under some additional assumptions which usually hold for orthobases of algebraic polynomial and is almost as simple as the linear projective estimate. Thus, it takes a form convenient for the construction of an adaptive estimator, which does not usea-priori information about the smoothness of the density. The adaptive estimation problem is briefly discussed and an unknown density fitting by Jacobi polynomials is investigated more explicitly.     

Application of fast spherical Fourier transform to density estimation

This paper is on density estimation on the 2-sphere, S², using the orthogonal series estimator corresponding to spherical harmonics. In the standard approach of truncating the Fourier series of the empirical density, the Fourier transform is replaced with a version of the discrete fast spherical Fourier transform, as developed by Driscoll and Healy. The fast transform only applies to quantitative data on a regular grid. We will apply a kernel operator to the empirical density, to produce a function whose values at the vertices of such a grid will be the basis for the density estimation. The proposed estimation procedure also contains a deconvolution step, in order to reduce the bias introduced by the initial kernel operator. The main issue is to find necessary conditions on the involved discretization and the bandwidth of the kernel operator, to preserve the rate of convergence that can be achieved by the usual computationally intensive Fourier transform. Density estimation is considered in L²(S²) and more generally in Sobolev spaces H_v(S²), any v?0, with the regularity assumption that the probability density to be estimated belongs to H_s(S²) for some s>v. The proposed technique to estimate the Fourier transform of an unknown density keeps computing cost down to order O(n), where n denotes the sample size.     

Estimating the error distribution in nonparametric multiple regression with applications to model testing

In this paper we consider the estimation of the error distribution in a heteroscedastic nonparametric regression model with multivariate covariates. As estimator we consider the empirical distribution function of residuals, which are obtained from multivariate local polynomial fits of the regression and variance functions, respectively. Weak convergence of the empirical residual process to a Gaussian process is proved. We also consider various applications for testing model assumptions in nonparametric multiple regression. The model tests obtained are able to detect local alternatives that converge to zero at an n^−1/2-rate, independent of the covariate dimension. We consider in detail a test for additivity of the regression function.     

Nonlinear weighted least squares estimation of a three-parameter Weibull density with a nonparametric start

This paper is concerned with the parameter estimation problem for the three-parameter Weibull density which is widely employed as a model in reliability and lifetime studies. Our approach is a combination of nonparametric and parametric methods. The basic idea is to start with an initial nonparametric density estimate which needs to be as good as possible, and then apply the nonlinear least squares method to estimate the unknown parameters. As a main result, a theorem on the existence of the least squares estimate is obtained. Some simulations are given to show that our approach is satisfactory if the initial density is of good enough quality.     

L p-consistency of multivariate density estimates

L _p notion of the weak, mean, and strong consistency of the kernel method of multivariate density estimation is proposed and studied. The results expand, unify, or generalize most known results in the literature. Rates of convergence in mean and strongL _p-consistencies are presented.     

Flexible modeling based on copulas in nonparametric median regression

Consider the model Y=m(X)+ε, where m(⋅)=med(Y|⋅) is unknown but smooth. It is often assumed that ε and X are independent. However, in practice this assumption is violated in many cases. In this paper we propose modeling the dependence between ε and X by means of a copula model, i.e. (ε,X)∼C_θ(F_ε(⋅),F_X(⋅)), where C_θ is a copula function depending on an unknown parameter θ, and F_ε and F_X are the marginals of ε and X. Since many parametric copula families contain the independent copula as a special case, the so-obtained regression model is more flexible than the ‘classical’ regression model.We estimate the parameter θ via a pseudo-likelihood method and prove the asymptotic normality of the estimator, based on delicate empirical process theory. We also study the estimation of the conditional distribution of Y given X. The procedure is illustrated by means of a simulation study, and the method is applied to data on food expenditures in households.     

Robust estimation in a nonlinear cointegration model

This paper considers the nonparametric M-estimator in a nonlinear cointegration type model. The local time density argument, which was developed by Phillips and Park (1998) [6] and Wang and Phillips (2009) [9], is applied to establish the asymptotic theory for the nonparametric M-estimator. The weak consistency and the asymptotic distribution of the proposed estimator are established under mild conditions. Meanwhile, the asymptotic distribution of the local least squares estimator and the local least absolute distance estimator can be obtained as applications of our main results. Furthermore, an iterated procedure for obtaining the nonparametric M-estimator and a cross-validation bandwidth selection method are discussed, and some numerical examples are provided to show that the proposed methods perform well in the finite sample case.     

Gaussian model selection

Our purpose in this paper is to provide a general approach to model selection via penalization for Gaussian regression and to develop our point of view about this subject. The advantage and importance of model selection come from the fact that it provides a suitable approach to many different types of problems, starting from model selection per se (among a family of parametric models, which one is more suitable for the data at hand), which includes for instance variable selection in regression models, to nonparametric estimation, for which it provides a very powerful tool that allows adaptation under quite general circumstances. Our approach to model selection also provides a natural connection between the parametric and nonparametric points of view and copes naturally with the fact that a model is not necessarily true. The method is based on the penalization of a least squares criterion which can be viewed as a generalization of Mallows’C _p . A large part of our efforts will be put on choosing properly the list of models and the penalty function for various estimation problems like classical variable selection or adaptive estimation for various types of l _p -bodies. Received February 1, 1999 / final version received January 10, 2001?Published online April 3, 2001     

Sharp adaptive estimation of quadratic functionals

Estimation of a quadratic functional of a function observed in the Gaussian white noise model is considered. A data-dependent method for choosing the amount of smoothing is given. The method is based on comparing certain quadratic estimators with each other. It is shown that the method is asymptotically sharp or nearly sharp adaptive simultaneously for the “regular” and “irregular” region. We consider l_p bodies and construct bounds for the risk of the estimator which show that for p=4 the estimator is exactly optimal and for example when p ∈[3,100], then the upper bound is at most 1.055 times larger than the lower bound. We show the connection of the estimator to the theory of optimal recovery. The estimator is a calibration of an estimator which is nearly minimax optimal among quadratic estimators. Writing of this article was financed by Deutsche Forschungsgemeinschaft under project MA1026/6-2, CIES, France, and Jenny and AnttiWihuri Foundation.     

On the risk of estimates for block decreasing densities

A density f=f(x₁,…,x_d) on [0,∞)^d is block decreasing if for each j∈{1,…,d}, it is a decreasing function of x_j, when all other components are held fixed. Let us consider the class of all block decreasing densities on [0,1]^d bounded by B. We shall study the minimax risk over this class using n i.i.d. observations, the loss being measured by the L₁ distance between the estimate and the true density. We prove that if S=log(1+B), lower bounds for the risk are of the form C(S^d/n)^1/(d+2), where C is a function of d only. We also prove that a suitable histogram with unequal bin widths as well as a variable kernel estimate achieve the optimal multivariate rate. We present a procedure for choosing all parameters in the kernel estimate automatically without loosing the minimax optimality, even if B and the support of f are unknown.     

Single-index quantile regression

Nonparametric quantile regression with multivariate covariates is a difficult estimation problem due to the “curse of dimensionality”. To reduce the dimensionality while still retaining the flexibility of a nonparametric model, we propose modeling the conditional quantile by a single-index function , where a univariate link function g₀(⋅) is applied to a linear combination of covariates , often called the single-index. We introduce a practical algorithm where the unknown link function g₀(⋅) is estimated by local linear quantile regression and the parametric index is estimated through linear quantile regression. Large sample properties of estimators are studied, which facilitate further inference. Both the modeling and estimation approaches are demonstrated by simulation studies and real data applications.     

Local likelihood estimation for nonstationary random fields

We develop a weighted local likelihood estimate for the parameters that govern the local spatial dependency of a locally stationary random field. The advantage of this local likelihood estimate is that it smoothly downweights the influence of faraway observations, works for irregular sampling locations, and when designed appropriately, can trade bias and variance for reducing estimation error. This paper starts with an exposition of our technique on the problem of estimating an unknown positive function when multiplied by a stationary random field. This example gives concrete evidence of the benefits of our local likelihood as compared to unweighted local likelihoods. We then discuss the difficult problem of estimating a bandwidth parameter that controls the amount of influence from distant observations. Finally we present a simulation experiment for estimating the local smoothness of a local Matérn random field when observing the field at random sampling locations in [0,1]². The local Matérn is a fully nonstationary random field, has a closed form covariance, can attain any degree of differentiability or Hölder smoothness and behaves locally like a stationary Matérn. We include an appendix that proves the positive definiteness of this covariance function.     

Optimal prediction for linear regression with infinitely many parameters

The problem of optimal prediction in the stochastic linear regression model with infinitely many parameters is considered. We suggest a prediction method that outperforms asymptotically the ordinary least squares predictor. Moreover, if the random errors are Gaussian, the method is asymptotically minimax over ellipsoids in ?₂. The method is based on a regularized least squares estimator with weights of the Pinsker filter. We also consider the case of dynamic linear regression, which is important in the context of transfer function modeling.     

Statistical inference for panel data semiparametric partially linear regression models with heteroscedastic errors

We consider a panel data semiparametric partially linear regression model with an unknown parameter vector for the linear parametric component, an unknown nonparametric function for the nonlinear component, and a one-way error component structure which allows unequal error variances (referred to as heteroscedasticity). We develop procedures to detect heteroscedasticity and one-way error component structure, and propose a weighted semiparametric least squares estimator (WSLSE) of the parametric component in the presence of heteroscedasticity and/or one-way error component structure. This WSLSE is asymptotically more efficient than the usual semiparametric least squares estimator considered in the literature. The asymptotic properties of the WSLSE are derived. The nonparametric component of the model is estimated by the local polynomial method. Some simulations are conducted to demonstrate the finite sample performances of the proposed testing and estimation procedures. An example of application on a set of panel data of medical expenditures in Australia is also illustrated.     

Efficient estimation in conditional single-index regression

Semiparametric single-index regression involves an unknown finite-dimensional parameter and an unknown (link) function. We consider estimation of the parameter via the pseudo-maximum likelihood method. For this purpose we estimate the conditional density of the response given a candidate index and maximize the obtained likelihood. We show that this technique of adaptation yields an asymptotically efficient estimator: it has minimal variance among all estimators.     

Nonconcave penalized inverse regression in single-index models with high dimensional predictors

In this paper we aim to estimate the direction in general single-index models and to select important variables simultaneously when a diverging number of predictors are involved in regressions. Towards this end, we propose the nonconcave penalized inverse regression method. Specifically, the resulting estimation with the SCAD penalty enjoys an oracle property in semi-parametric models even when the dimension, p_n, of predictors goes to infinity. Under regularity conditions we also achieve the asymptotic normality when the dimension of predictor vector goes to infinity at the rate of p_n=o(n^1/3) where n is sample size, which enables us to construct confidence interval/region for the estimated index. The asymptotic results are augmented by simulations, and illustrated by analysis of an air pollution dataset.     

On minimax identification of nonparametric autoregressive models

We consider the problem of nonparametric identification for a multi-dimensional functional autoregression y _t = f(y _{t −1}, …,y _t−d ) + e _t on the basis of N observations of y _t . In the case when the unknown nonlinear function f belongs to the Barron class, we propose an estimation algorithm which provides approximations of f with expected L ₂ accuracy O(N ^1/4ln^1/4 N). We also show that this approximation rate cannot be significantly improved. The proposed algorithms are “computationally efficient”– the total number of elementary computations necessary to complete the estimate grows polynomially with N. Received: 23 September 1997 / Revised version: 28 January 1999     

Asymptotic bias and variance for a general class of varying bandwidth density estimators

Summary We consider a general class of varying bandwidth estimators of a probability density function. The class includes the Abramson estimator, transformation kernel density estimator (TKDE), Jones transformation kernel density estimator (JTKDE), nearest neighbour type estimator (NN), Jones-Linton-Nielsen estimator (JLN), Taylor series approximations of TKDE (TTKDE) and Simpson's formula approximations of TKDE (STKDE). Each of these estimators needs a pilot estimator. Starting with an ordinary kernel estimator , it is possible to iterate and compute a sequence of estimates , using each estimate as a pilot estimator in the next step. The first main result is a formula for the bias order. If the bandwidths used in different steps have a common orderh=h(n), the bias of is of orderh ^2km ,k=1, ...,t. Hereh ^m is the bias order of the ideal estimator (defined by using the unknownf as pilot). The second main result is a recursive formula for the leading bias and stochastic terms in an asymptotic expansion of the density estimates. Ifm<, it is possible to make asymptotically equivalent to the ideal estimator.     

Nonparametric regression estimation with general parametric error covariance

The asymptotic distribution for the local linear estimator in nonparametric regression models is established under a general parametric error covariance with dependent and heterogeneously distributed regressors. A two-step estimation procedure that incorporates the parametric information in the error covariance matrix is proposed. Sufficient conditions for its asymptotic normality are given and its efficiency relative to the local linear estimator is established. We give examples of how our results are useful in some recently studied regression models. A Monte Carlo study confirms the asymptotic theory predictions and compares our estimator with some recently proposed alternative estimation procedures.     

Asymptotic properties of nonparametric curve estimates

We consider a class of nonparametric estimators for the regression functionm(t) in the model:y _i =m(t _i ) + _i , 1 i n, t _i [0, 1], which are linear in the observationsy _i . Several limit theorems concerning local and global stochastic and a.s. convergence and limit distributions are given.