Nonparametric regression estimation with general parametric error covariance

The asymptotic distribution for the local linear estimator in nonparametric regression models is established under a general parametric error covariance with dependent and heterogeneously distributed regressors. A two-step estimation procedure that incorporates the parametric information in the error covariance matrix is proposed. Sufficient conditions for its asymptotic normality are given and its efficiency relative to the local linear estimator is established. We give examples of how our results are useful in some recently studied regression models. A Monte Carlo study confirms the asymptotic theory predictions and compares our estimator with some recently proposed alternative estimation procedures.     

Berry-Esseen bounds for wavelet estimator in a regression model with linear process errors

In this paper, we derive the Berry-Esseen bounds of the wavelet estimator for a nonparametric regression model with linear process errors generated by φ-mixing sequences. As application, by the suitable choice of some constants, the convergence rate O(n^−1/6) of uniformly asymptotic normality of the wavelet estimator is obtained. Our results generalize some known results in the literature.     

Variable selection for semiparametric varying coefficient partially linear errors-in-variables models

This paper focuses on the variable selections for semiparametric varying coefficient partially linear models when the covariates in the parametric and nonparametric components are all measured with errors. A bias-corrected variable selection procedure is proposed by combining basis function approximations with shrinkage estimations. With appropriate selection of the tuning parameters, the consistency of the variable selection procedure and the oracle property of the regularized estimators are established. A simulation study and a real data application are undertaken to evaluate the finite sample performance of the proposed method.     

Asymptotic efficiency of ridge estimator in linear and semiparametric linear models

The linear model with a growing number of predictors arises in many contemporary scientific endeavor. In this article, we consider the commonly used ridge estimator in linear models. We propose analyzing the ridge estimator for a finite sample size n and a growing dimension p. The existence and asymptotic normality of the ridge estimator are established under some regularity conditions when p→∞. It also occurs that a strictly linear model is inadequate when some of the relations are believed to be of certain linear form while others are not easily parameterized, and thus a semiparametric partial linear model is considered. For these semiparametric partial linear models with p>n, we develop a procedure to estimate the linear coefficients as if the nonparametric part is not present. The asymptotic efficiency of the proposed estimator for the linear component is studied for p→∞. It is shown that the proposed estimator of the linear component asymptotically performs very well.     

Censored multiple regression by the method of average derivatives

This paper proposes a technique [termed censored average derivative estimation (CADE)] for studying estimation of the unknown regression function in nonparametric censored regression models with randomly censored samples. The CADE procedure involves three stages: firstly-transform the censored data into synthetic data or pseudo-responses using the inverse probability censoring weighted (IPCW) technique, secondly estimate the average derivatives of the regression function, and finally approximate the unknown regression function by an estimator of univariate regression using techniques for one-dimensional nonparametric censored regression. The CADE provides an easily implemented methodology for modelling the association between the response and a set of predictor variables when data are randomly censored. It also provides a technique for “dimension reduction” in nonparametric censored regression models. The average derivative estimator is shown to be root-n consistent and asymptotically normal. The estimator of the unknown regression function is a local linear kernel regression estimator and is shown to converge at the optimal one-dimensional nonparametric rate. Monte Carlo experiments show that the proposed estimators work quite well.     

Statistical inference for panel data semiparametric partially linear regression models with heteroscedastic errors

We consider a panel data semiparametric partially linear regression model with an unknown parameter vector for the linear parametric component, an unknown nonparametric function for the nonlinear component, and a one-way error component structure which allows unequal error variances (referred to as heteroscedasticity). We develop procedures to detect heteroscedasticity and one-way error component structure, and propose a weighted semiparametric least squares estimator (WSLSE) of the parametric component in the presence of heteroscedasticity and/or one-way error component structure. This WSLSE is asymptotically more efficient than the usual semiparametric least squares estimator considered in the literature. The asymptotic properties of the WSLSE are derived. The nonparametric component of the model is estimated by the local polynomial method. Some simulations are conducted to demonstrate the finite sample performances of the proposed testing and estimation procedures. An example of application on a set of panel data of medical expenditures in Australia is also illustrated.     

Nonparametric methods for unbalanced multivariate data and many factor levels

We propose different nonparametric tests for multivariate data and derive their asymptotic distribution for unbalanced designs in which the number of factor levels tends to infinity (large a, small n_i case). Quasi gratis, some new parametric multivariate tests suitable for the large a asymptotic case are also obtained. Finite sample performances are investigated and compared in a simulation study. The nonparametric tests are based on separate rankings for the different variables. In the presence of outliers, the proposed nonparametric methods have better power than their parametric counterparts. Application of the new tests is demonstrated using data from plant pathology.     

Asymptotic distributions of two “synthetic data” estimators for censored single-index models

The censored single-index model provides a flexible way for modelling the association between a response and a set of predictor variables when the response variable is randomly censored and the link function is unknown. It presents a technique for “dimension reduction” in semiparametric censored regression models and generalizes the existing accelerated failure time models for survival analysis. This paper proposes two methods for estimation of single-index models with randomly censored samples. We first transform the censored data into synthetic data or pseudo-responses unbiasedly, then obtain estimates of the index coefficients by the rOPG or rMAVE procedures of Xia (2006) [1]. Finally, we estimate the unknown nonparametric link function using techniques for univariate censored nonparametric regression. The estimators for the index coefficients are shown to be root-n consistent and asymptotically normal. In addition, the estimator for the unknown regression function is a local linear kernel regression estimator and can be estimated with the same efficiency as the parameters are known. Monte Carlo simulations are conducted to illustrate the proposed methodologies.     

Statistical inference in a panel data semiparametric regression model with serially correlated errors

We consider a panel data semiparametric partially linear regression model with an unknown vector β of regression coefficients, an unknown nonparametric function g(·) for nonlinear component, and unobservable serially correlated errors. The correlated errors are modeled by a vector autoregressive process which involves a constant intraclass correlation. Applying the pilot estimators of β and g(·), we construct estimators of the autoregressive coefficients, the intraclass correlation and the error variance, and investigate their asymptotic properties. Fitting the error structure results in a new semiparametric two-step estimator of β, which is shown to be asymptotically more efficient than the usual semiparametric least squares estimator in terms of asymptotic covariance matrix. Asymptotic normality of this new estimator is established, and a consistent estimator of its asymptotic covariance matrix is presented. Furthermore, a corresponding estimator of g(·) is also provided. These results can be used to make asymptotically efficient statistical inference. Some simulation studies are conducted to illustrate the finite sample performances of these proposed estimators.     

Robust estimation in a nonlinear cointegration model

This paper considers the nonparametric M-estimator in a nonlinear cointegration type model. The local time density argument, which was developed by Phillips and Park (1998) [6] and Wang and Phillips (2009) [9], is applied to establish the asymptotic theory for the nonparametric M-estimator. The weak consistency and the asymptotic distribution of the proposed estimator are established under mild conditions. Meanwhile, the asymptotic distribution of the local least squares estimator and the local least absolute distance estimator can be obtained as applications of our main results. Furthermore, an iterated procedure for obtaining the nonparametric M-estimator and a cross-validation bandwidth selection method are discussed, and some numerical examples are provided to show that the proposed methods perform well in the finite sample case.     

Estimating the error distribution in nonparametric multiple regression with applications to model testing

In this paper we consider the estimation of the error distribution in a heteroscedastic nonparametric regression model with multivariate covariates. As estimator we consider the empirical distribution function of residuals, which are obtained from multivariate local polynomial fits of the regression and variance functions, respectively. Weak convergence of the empirical residual process to a Gaussian process is proved. We also consider various applications for testing model assumptions in nonparametric multiple regression. The model tests obtained are able to detect local alternatives that converge to zero at an n^−1/2-rate, independent of the covariate dimension. We consider in detail a test for additivity of the regression function.     

Statistical inference in partially-varying-coefficient single-index model

Consider a varying-coefficient single-index model which consists of two parts: the linear part with varying coefficients and the nonlinear part with a single-index structure, and are hence termed as varying-coefficient single-index models. This model includes many important regression models such as single-index models, partially linear single-index models, varying-coefficient model and varying-coefficient partially linear models as special examples. In this paper, we mainly study estimating problems of the varying-coefficient vector, the nonparametric link function and the unknown parametric vector describing the single-index in the model. A stepwise approach is developed to obtain asymptotic normality estimators of the varying-coefficient vector and the parametric vector, and estimators of the nonparametric link function with a convergence rate. The consistent estimator of the structural error variance is also obtained. In addition, asymptotic pointwise confidence intervals and confidence regions are constructed for the varying coefficients and the parametric vector. The bandwidth selection problem is also considered. A simulation study is conducted to evaluate the proposed methods, and real data analysis is also used to illustrate our methods.     

Functional semiparametric partially linear model with autoregressive errors

In this paper, we introduce a functional semiparametric model, where a real-valued random variable is explained by the sum of a unknown linear combination of the components of a multivariate random variable and an unknown transformation of a functional random variable. The errors can be autocorrelated. We focus here on the parametric estimation of the coefficients in the linear combination. First, we use a nonparametric kernel method to remove the effect of the functional explanatory variable. Then, we use generalized least squares approach to obtain an estimator of these coefficients. Under some technical assumptions, we prove consistency and asymptotic normality of our estimator. Finally, we present Monte Carlo simulations that illustrate these characteristics.     

Density testing in a contaminated sample

We study non-parametric tests for checking parametric hypotheses about a multivariate density f of independent identically distributed random vectors Z₁,Z₂,… which are observed under additional noise with density ψ. The tests we propose are an extension of the test due to Bickel and Rosenblatt [On some global measures of the deviations of density function estimates, Ann. Statist. 1 (1973) 1071-1095] and are based on a comparison of a nonparametric deconvolution estimator and the smoothed version of a parametric fit of the density f of the variables of interest Z_i. In an example the loss of efficiency is highlighted when the test is based on the convolved (but observable) density g=f*ψ instead on the initial density of interest f.     

Nonconcave penalized inverse regression in single-index models with high dimensional predictors

In this paper we aim to estimate the direction in general single-index models and to select important variables simultaneously when a diverging number of predictors are involved in regressions. Towards this end, we propose the nonconcave penalized inverse regression method. Specifically, the resulting estimation with the SCAD penalty enjoys an oracle property in semi-parametric models even when the dimension, p_n, of predictors goes to infinity. Under regularity conditions we also achieve the asymptotic normality when the dimension of predictor vector goes to infinity at the rate of p_n=o(n^1/3) where n is sample size, which enables us to construct confidence interval/region for the estimated index. The asymptotic results are augmented by simulations, and illustrated by analysis of an air pollution dataset.     

Testing independence in nonparametric regression

We propose a new test for independence of error and covariate in a nonparametric regression model. The test statistic is based on a kernel estimator for the L₂-distance between the conditional distribution and the unconditional distribution of the covariates. In contrast to tests so far available in literature, the test can be applied in the important case of multivariate covariates. It can also be adjusted for models with heteroscedastic variance. Asymptotic normality of the test statistic is shown. Simulation results and a real data example are presented.     

Optimal global rate of convergence in nonparametric regression with left-truncated and right-censored data

In this paper we consider nonparametric regression with left-truncated and right-censored data. An estimator of the regression function is developed when censoring and truncation are independent of covariates and the response. The estimation is based on the product limit estimator of the response variable. Under certain conditions, the L₂ rate of convergence of the estimated regression function is obtained when tensor products of B-splines are used.     

Difference based estimation for partially linear regression models with measurement errors

This paper is concerned with the estimating problem of the partially linear regression models where the linear covariates are measured with additive errors. A difference based estimation is proposed to estimate the parametric component. We show that the resulting estimator is asymptotically unbiased and achieves the semiparametric efficiency bound if the order of the difference tends to infinity. The asymptotic normality of the resulting estimator is established as well. Compared with the corrected profile least squares estimation, the proposed procedure avoids the bandwidth selection. In addition, the difference based estimation of the error variance is also considered. For the nonparametric component, the local polynomial technique is implemented. The finite sample properties of the developed methodology is investigated through simulation studies. An example of application is also illustrated.     

Smooth estimation of survival and density functions for a stationary associated process using Poisson weights

Let {X_n,n≥1} be a sequence of stationary non-negative associated random variables with common marginal density f(x). Here we use the empirical survival function as studied in Bagai and Prakasa Rao (1991) and apply the smoothing technique proposed by Gawronski (1980) (see also Chaubey and Sen, 1996) in proposing a smooth estimator of the density function f and that of the corresponding survival function. Some asymptotic properties of the resulting estimators, similar to those obtained in Chaubey and Sen (1996) for the i.i.d. case, are derived. A simulation study has been carried out to compare the new estimator to the kernel estimator of a density function given in Bagai and Prakasa Rao (1996) and the estimator in Buch-Larsen et al. (2005).