Estimating the error distribution in nonparametric multiple regression with applications to model testing

In this paper we consider the estimation of the error distribution in a heteroscedastic nonparametric regression model with multivariate covariates. As estimator we consider the empirical distribution function of residuals, which are obtained from multivariate local polynomial fits of the regression and variance functions, respectively. Weak convergence of the empirical residual process to a Gaussian process is proved. We also consider various applications for testing model assumptions in nonparametric multiple regression. The model tests obtained are able to detect local alternatives that converge to zero at an n^−1/2-rate, independent of the covariate dimension. We consider in detail a test for additivity of the regression function.     

Spline-based sieve maximum likelihood estimation in the partly linear model under monotonicity constraints

We study a spline-based likelihood method for the partly linear model with monotonicity constraints. We use monotone B-splines to approximate the monotone nonparametric function and apply the generalized Rosen algorithm to compute the estimators jointly. We show that the spline estimator of the nonparametric component achieves the possible optimal rate of convergence under the smooth assumption and that the estimator of the regression parameter is asymptotically normal and efficient. Moreover, a spline-based semiparametric likelihood ratio test is established to make inference of the regression parameter. Also an observed profile information method to consistently estimate the standard error of the spline estimator of the regression parameter is proposed. A simulation study is conducted to evaluate the finite sample performance of the proposed method. The method is illustrated by an air pollution study.     

Estimating and Visualizing Conditional Densities

Abstract

We consider the kernel estimator of conditional density and derive its asymptotic bias, variance, and mean-square error. Optimal bandwidths (with respect to integrated mean-square error) are found and it is shown that the convergence rate of the density estimator is order n ^–2/3. We also note that the conditional mean function obtained from the estimator is equivalent to a kernel smoother. Given the undesirable bias properties of kernel smoothers, we seek a modified conditional density estimator that has mean equivalent to some other nonparametric regression smoother with better bias properties. It is also shown that our modified estimator has smaller mean square error than the standard estimator in some commonly occurring situations. Finally, three graphical methods for visualizing conditional density estimators are discussed and applied to a data set consisting of maximum daily temperatures in Melbourne, Australia.     

Optimal testing for additivity in multiple nonparametric regression

We consider the problem of testing for additivity in the standard multiple nonparametric regression model. We derive optimal (in the minimax sense) non- adaptive and adaptive hypothesis testing procedures for additivity against the composite nonparametric alternative that the response function involves interactions of second or higher orders separated away from zero in L ²([0, 1] ^d )-norm and also possesses some smoothness properties. In order to shed some light on the theoretical results obtained, we carry out a wide simulation study to examine the finite sample performance of the proposed hypothesis testing procedures and compare them with a series of other tests for additivity available in the literature.     

Statistical inference in a panel data semiparametric regression model with serially correlated errors

We consider a panel data semiparametric partially linear regression model with an unknown vector β of regression coefficients, an unknown nonparametric function g(·) for nonlinear component, and unobservable serially correlated errors. The correlated errors are modeled by a vector autoregressive process which involves a constant intraclass correlation. Applying the pilot estimators of β and g(·), we construct estimators of the autoregressive coefficients, the intraclass correlation and the error variance, and investigate their asymptotic properties. Fitting the error structure results in a new semiparametric two-step estimator of β, which is shown to be asymptotically more efficient than the usual semiparametric least squares estimator in terms of asymptotic covariance matrix. Asymptotic normality of this new estimator is established, and a consistent estimator of its asymptotic covariance matrix is presented. Furthermore, a corresponding estimator of g(·) is also provided. These results can be used to make asymptotically efficient statistical inference. Some simulation studies are conducted to illustrate the finite sample performances of these proposed estimators.     

Testing independence in nonparametric regression

We propose a new test for independence of error and covariate in a nonparametric regression model. The test statistic is based on a kernel estimator for the L₂-distance between the conditional distribution and the unconditional distribution of the covariates. In contrast to tests so far available in literature, the test can be applied in the important case of multivariate covariates. It can also be adjusted for models with heteroscedastic variance. Asymptotic normality of the test statistic is shown. Simulation results and a real data example are presented.     

Statistical inference for panel data semiparametric partially linear regression models with heteroscedastic errors

We consider a panel data semiparametric partially linear regression model with an unknown parameter vector for the linear parametric component, an unknown nonparametric function for the nonlinear component, and a one-way error component structure which allows unequal error variances (referred to as heteroscedasticity). We develop procedures to detect heteroscedasticity and one-way error component structure, and propose a weighted semiparametric least squares estimator (WSLSE) of the parametric component in the presence of heteroscedasticity and/or one-way error component structure. This WSLSE is asymptotically more efficient than the usual semiparametric least squares estimator considered in the literature. The asymptotic properties of the WSLSE are derived. The nonparametric component of the model is estimated by the local polynomial method. Some simulations are conducted to demonstrate the finite sample performances of the proposed testing and estimation procedures. An example of application on a set of panel data of medical expenditures in Australia is also illustrated.     

A bias corrected nonparametric regression estimator

In this article, we propose a new method of bias reduction in nonparametric regression estimation. The proposed new estimator has asymptotic bias order h⁴, where h is a smoothing parameter, in contrast to the usual bias order h² for the local linear regression. In addition, the proposed estimator has the same order of the asymptotic variance as the local linear regression. Our proposed method is closely related to the bias reduction method for kernel density estimation proposed by Chung and Lindsay (2011). However, our method is not a direct extension of their density estimate, but a totally new one based on the bias cancelation result of their proof.     

Confidence Intervals of Variance Functions in Generalized Linear Model

In this paper we introduce an appealing nonparametric method for estimating variance and conditional variance functions in generalized linear models （GLMs）, when designs are fixed points and random variables respectively, Bias-corrected confidence bands are proposed for the （conditional） variance by local linear smoothers. Nonparametric techniques are developed in deriving the bias-corrected confidence intervals of the （conditional） variance. The asymptotic distribution of the proposed estimator is established and show that the bias-corrected confidence bands asymptotically have the correct coverage properties. A small simulation is performed when unknown regression parameter is estimated by nonparametric quasi-likelihood. The results are also applicable to nonparamctric autoregressive times series model with heteroscedastic conditional variance.     

On the cusum of squares test for variance change in nonstationary and nonparametric time series models

In this paper we consider the problem of testing for a variance change in nonstationary and nonparametric time series models. The models under consideration are the unstable AR(q) model and the fixed design nonparametric regression model with a strong mixing error process. In order to perform a test, we employ the cusum of squares test introduced by Inclán and Tiao (1994,J. Amer. Statist. Assoc.,89, 913–923). It is shown that the limiting distribution of the test statistic is the sup of a standard Brownian bridge as seen in iid random samples. Simulation results are provided for illustration.     

Stahel-Donoho kernel estimation for fixed design nonparametric regression models

This paper reports a robust kernel estimation for fixed design nonparametric regression models. A Stahel-Donoho kernel estimation is introduced, in which the weight functions depend on both the depths of data and the distances between the design points and the estimation points. Based on a local approximation, a computational technique is given to approximate to the incomputable depths of the errors. As a result the new estimator is computationally efficient. The proposed estimator attains a high breakdown point and has perfect asymptotic behaviors such as the asymptotic normality and convergence in the mean squared error. Unlike the depth-weighted estimator for parametric regression models, this depth-weighted nonparametric estimator has a simple variance structure and then we can compare its efficiency with the original one. Some simulations show that the new method can smooth the regression estimation and achieve some desirable balances between robustness and efficiency.     

Locally efficient estimation of regression parameters using current status data

In biostatistics applications interest often focuses on the estimation of the distribution of a time-variable T. If one only observes whether or not T exceeds an observed monitoring time C, then the data structure is called current status data, also known as interval censored data, case I. We consider this data structure extended to allow the presence of both time-independent covariates and time-dependent covariate processes that are observed until the monitoring time. We assume that the monitoring process satisfies coarsening at random.Our goal is to estimate the regression parameter β of the regression model T=Zβ+ε. The curse of dimensionality implies no globally efficient nonparametric estimator with good practical performance at moderate sample sizes exists. We present an estimator of the parameter β that attains the semiparametric efficiency bound if we correctly specify (a) a model for the monitoring mechanism and (b) a lower-dimensional model for the conditional distribution of T given the covariates. In addition, our estimator is robust to model misspecification. If only (a) is correctly specified, the estimator remains consistent and asymptotically normal. We conclude with a simulation experiment and a data analysis.     

A simple test for the parametric form of the variance function in nonparametric regression

In this paper a new test for the parametric form of the variance function in the common nonparametric regression model is proposed which is applicable under very weak smoothness assumptions. The new test is based on an empirical process formed from pseudo residuals, for which weak convergence to a Gaussian process can be established. In the special case of testing for homoscedasticity the limiting process is essentially a Brownian bridge, such that critical values are easily available. The new procedure has three main advantages. First, in contrast to many other methods proposed in the literature, it does not depend directly on a smoothing parameter. Secondly, it can detect local alternatives converging to the null hypothesis at a rate n ^−1/2. Thirdly, in contrast to most of the currently available tests, it does not require strong smoothness assumptions regarding the regression and variance function. We also present a simulation study and compare the tests with the procedures that are currently available for this problem and require the same minimal assumptions.     

Testing Serial Correlation in Semiparametric Varying-Coefficient Partially Linear EV Models

This paper studies estimation and serial correlation test of a semiparametric varying-coefficient partially linear EV model of the form Y = X^Tβ ＋Z^Tα（T） ＋ε,ξ = X ＋ η with the identifying condition E[（ε,η^T）^T] =0, Cov[（ε,η^T）^T] = σ^2Ip＋1. The estimators of interested regression parameters /3 , and the model error variance σ2, as well as the nonparametric components α（T）, are constructed. Under some regular conditions, we show that the estimators of the unknown vector β and the unknown parameter σ2 are strongly consistent and asymptotically normal and that the estimator of α（T） achieves the optimal strong convergence rate of the usual nonparametric regression. Based on these estimators and asymptotic properties, we propose the VN,p test statistic and empirical log-likelihood ratio statistic for testing serial correlation in the model. The proposed statistics are shown to have asymptotic normal or chi-square distributions under the null hypothesis of no serial correlation. Some simulation studies are conducted to illustrate the finite sample performance of the proposed tests.     

Monotone nonparametric regression with random design

In this paper we study the nonparametric least squares estimator of a regression function in a random design setting under the constraint that this function is monotone, say, nonincreasing. The errors are not assumed conditionally i.i.d. given the observation points. In particular, this includes the case of conditional heteroscedasticity and the case of the current status model. The -error is shown to be of order n ^−p/3 and asymptotically Gaussian with explicit asymptotic mean and variance.      

Generalized profile LSE in varying-coefficient partially linear models with measurement errors

This paper is concerned with the estimating problem of a semiparametric varying-coefficient partially linear errors-in-variables model Yi=Xτiβ+Zτiα(Ui)+εi , Wi=Xi+ξi,i=1, ··· , n. Due to measurement errors, the usual profile least square estimator of the parametric component, local polynomial estimator of the nonparametric component and profile least squares based estimator of the error variance are biased and inconsistent. By taking the measurement errors into account we propose a generalized profile least squares estimator for the parametric component and show it is consistent and asymptotically normal. Correspondingly, the consistent estimation of the nonparametric component and error variance are proposed as well. These results may be used to make asymptotically valid statistical inferences. Some simulation studies are conducted to illustrate the finite sample performance of these proposed estimations.     

Permutation tests using least distance estimator in the multivariate regression model

This paper proposes two permutation tests based on the least distance estimator in a multivariate regression model. One is a type of t test statistic using the bootstrap method, and the other is a type of F test statistic using the sum of distances between observed and predicted values under the full and reduced models. We conducted a simulation study to compare the power of the proposed permutation tests with that of the parametric tests based on the least squares estimator for three types of hypotheses in several error distributions. The results indicate that the power of the proposed permutation tests is greater than that of the parametric tests when the error distribution is skewed like the Wishart distribution, has a heavy tail like the Cauchy distribution, or has outliers.     

�̶�ЧӦ������ݲ�������ģ�͵ļ�Ȩ����LSDV����

This paper concerns with the estimation of a fixed effects panel data partially linear regression model with the idiosyncratic errors being an autoregressive process. For fixed effects short time series panel data, the commonly used autoregressive error structure fitting method will not result in a consistent estimator of the autoregressive coefficients. Here we propose an alternative estimation and show that the resulting estimator of the autoregressive coefficients is consistent and this method is workable for any order autoregressive error structure. Moreover, combining the B-spline approximation, profile least squares dummy variable (PLSDV) technique and consistently estimated the autoregressive error structure, we develop a weighted PLSDV estimator for the parametric component and a weighted B-spline series (BS) estimator for the nonparametric component. The weighted PLSDV estimator is shown to be asymptotically normal and more asymptotically efficient than the one which ignores the error autoregressive structure. In addition, this paper derives the asymptotic bias of the weighted BS estimator and establish its asymptotic normality as well. Simulation studies and an example of application are conducted to illustrate the finite sample performance of the proposed procedures.