非参数回归函数估计的渐近正态性

本文研究了独立或相依样本时非参数回归函数的Ｎａｄａｒａｙａ-Ｗａｔｓｏｎ估计，在简洁合理的条件下，证明了估计量的渐近正态性．获得的结论可在时间序列分析中得到应用．     

Semiparametric and Additive Model Selection Using an Improved Akaike Information Criterion

Abstract

An improved AIC-based criterion is derived for model selection in general smoothing-based modeling, including semiparametric models and additive models. Examples are provided of applications to goodness-of-fit, smoothing parameter and variable selection in an additive model and semiparametric models, and variable selection in a model with a nonlinear function of linear terms.     

Efficient estimation of functional-coefficient regression models with different smoothing variables

In this article,a procedure for estimating the coefficient functions on the functional-coefficient regression models with different smoothing variables in different coefficient functions is defined.First step,by the local linear technique and the averaged method,the initial estimates of the coefficient functions are given.Second step,based on the initial estimates,the efficient estimates of the coefficient functions are proposed by a one-step back-fitting procedure.The efficient estimators share the same asymptotic normalities as the local linear estimators for the functional-coefficient models with a single smoothing variable in different functions.Two simulated examples show that the procedure is effective.     

Large and moderate deviations principles for kernel estimators of the multivariate regression

In this paper, we prove large deviations principle for the Nadaraya-Watson estimator and for the semi-recursive kernel estimator of the regression in the multidimensional case. Under suitable conditions, we show that the rate function is a good rate function. We thus generalize the results already obtained in the one-dimensional case for the Nadaraya-Watson estimator. Moreover, we give a moderate deviations principle for these two estimators. It turns out that the rate function obtained in the moderate deviations principle for the semi-recursive estimator is larger than the one obtained for the Nadaraya-Watson estimator.      

Bayesian hierarchical linear mixed models for additive smoothing splines

Bayesian hierarchical models have been used for smoothing splines, thin-plate splines, and L-splines. In analyzing high dimensional data sets, additive models and backfitting methods are often used. A full Bayesian analysis for such models may include a large number of random effects, many of which are not intuitive, so researchers typically use noninformative improper or nearly improper priors. We investigate propriety of the posterior for these cases. Our findings extend known results for normal linear mixed models to certain cases with Bayesian additive smoothing spline models. Supported by National Science Foundation grant SES-0351523 and by National Institutes of Health grants R01-CA100760 and R01-MH071418.     

A bias corrected nonparametric regression estimator

In this article, we propose a new method of bias reduction in nonparametric regression estimation. The proposed new estimator has asymptotic bias order h⁴, where h is a smoothing parameter, in contrast to the usual bias order h² for the local linear regression. In addition, the proposed estimator has the same order of the asymptotic variance as the local linear regression. Our proposed method is closely related to the bias reduction method for kernel density estimation proposed by Chung and Lindsay (2011). However, our method is not a direct extension of their density estimate, but a totally new one based on the bias cancelation result of their proof.     

Genetic algorithms for the selection of smoothing parameters in additive models

Summary  Additive models of the type y=f ₁(x ₁)+...+f _p(x _p)+ε where f _j , j=1,..,p, have unspecified functional form, are flexible statistical regression models which can be used to characterize nonlinear regression effects. One way of fitting additive models is the expansion in B-splines combined with penalization which prevents overfitting. The performance of this penalized B-spline (called P-spline) approach strongly depends on the choice of the amount of smoothing used for components f _j . In particular for higher dimensional settings this is a computationaly demanding task. In this paper we treat the problem of choosing the smoothing parameters for P-splines by genetic algorithms. In several simulation studies this approach is compared to various alternative methods of fitting additive models. In particular functions with different spatial variability are considered and the effect of constant respectively local adaptive smoothing parameters is evaluated.     

A Fast and Stable Updating Algorithm for Bivariate Nonparametric Curve Estimation

Abstract

An updating algorithm for bivariate local linear regression is proposed. Thereby, we assume a rectangular design and a polynomial kernel constrained to rectangular support as weight function. Results of univariate regression estimators are extended to the bivariate setting. The updates are performed in a way that most of the well-known numerical instabilities of a naive update implementation can be avoided. Some simulation results illustrate the properties of several algorithms with respect to computing time and numerical stability.     

Two-stage local M-estimation of additive models

This paper studies local M-estimation of the nonparametric components of additive models.A two-stage local M-estimation procedure is proposed for estimating the additive components and their derivatives.Under very mild conditions,the proposed estimators of each additive component and its derivative are jointly asymptotically normal and share the same asymptotic distributions as they would be if the other components were known.The established asymptotic results also hold for two particular local M-estimations:the local least squares and least absolute deviation estimations.However,for general two-stage local M-estimation with continuous and nonlinear ψ-functions,its implementation is time-consuming.To reduce the computational burden,one-step approximations to the two-stage local M-estimators are developed.The one-step estimators are shown to achieve the same effciency as the fully iterative two-stage local M-estimators,which makes the two-stage local M-estimation more feasible in practice.The proposed estimators inherit the advantages and at the same time overcome the disadvantages of the local least-squares based smoothers.In addition,the practical implementation of the proposed estimation is considered in details.Simulations demonstrate the merits of the two-stage local M-estimation,and a real example illustrates the performance of the methodology.     

Numerical discretization-based kernel type estimation methods for ordinary differential equation models

We consider the problem of parameter estimation in both linear and nonlinear ordinary differential equation(ODE) models. Nonlinear ODE models are widely used in applications. But their analytic solutions are usually not available. Thus regular methods usually depend on repetitive use of numerical solutions which bring huge computational cost. We proposed a new two-stage approach which includes a smoothing method(kernel smoothing or local polynomial fitting) in the first stage, and a numerical discretization method(Eulers discretization method, the trapezoidal discretization method,or the Runge–Kutta discretization method) in the second stage. Through numerical simulations, we find the proposed method gains a proper balance between estimation accuracy and computational cost.Asymptotic properties are also presented, which show the consistency and asymptotic normality of estimators under some mild conditions. The proposed method is compared to existing methods in term of accuracy and computational cost. The simulation results show that the estimators with local linear smoothing in the first stage and trapezoidal discretization in the second stage have the lowest average relative errors. We apply the proposed method to HIV dynamics data to illustrate the practicability of the estimator.     

Efficient estimation of a varying-coefficient partially linear binary regression model

This article considers a semiparametric varying-coefficient partially linear binary regression model. The semiparametric varying-coefficient partially linear regression binary model which is a generalization of binary regression model and varying-coefficient regression model that allows one to explore the possibly nonlinear effect of a certain covariate on the response variable. A Sieve maximum likelihood estimation method is proposed and the asymptotic properties of the proposed estimators are discussed. One of our main objects is to estimate nonparametric component and the unknowen parameters simultaneously. It is easier to compute, and the required computation burden is much less than that of the existing two-stage estimation method. Under some mild conditions, the estimators are shown to be strongly consistent. The convergence rate of the estimator for the unknown smooth function is obtained, and the estimator for the unknown parameter is shown to be asymptotically efficient and normally distributed. Simulation studies are carried out to investigate the performance of the proposed method.     

Estimation in additive cox models by marginal integration

Assuming an additive model on the covariate effect in proportional hazards regression, we consider the estimation of the component functions. The estimator is based on the marginal integration method. Then we use a new kind of nonparametric estimator as the pilot estimator of the marginal integration. The pilot estimator is constructed by an analogy to the two-sample problems and by appealing to the principles of local partial likelihood and local linear fitting. We derive the asymptotic distribution of the marginal integration estimator of the component functions. The result of a simulation study is also given.     

一类非参数回归函数估计的性质

设Z_(11),z_(12),…,Z_是在固定点(x_i,y_1),1≤≤n_1,1≤j≤n_2,的n_1n_2个观察值,适合模型 Z_(ij)=g(x_i,y_j)+ε_(ij),1≤i≤n_1,1≤j≤n_2。(1) 本文给出了g的一种估计并讨论了估计的性质。     

A bootstrap approach to model checking for linear models under length-biased data

In this paper, we propose two bootstrap-based model checking tests for a parametric linear model when data are affected by length-bias. These tests are based on the measure of the discrepancy between nonparametric and parametric estimators for the regression function when the data are drawn under a length-biased mechanism. We consider two different discrepancy measures: the supremum and the integral of the quadratic difference between the parametric and nonparametric estimators.     

Generalized Partially Linear Measurement Error Models

This article considers generalized partially linear models when the linear covariate is measured with additive error. We propose estimators of parameter and nonparametric function by using local linear regression, the SIMEX technique, and generalized estimating equation. The asymptotic normality of the estimators of the parameter, and bias and variance of the estimators of the nonparametric component are derived under appropriate assumptions. In addition, the generalization to clustered measurements is discussed. The approaches are used to the analysis of data from the Framingham Heart Study. A simulation experiment is conducted for an illustration.     

Jump-Preserving Regression and Smoothing using Local Linear Fitting: A Compromise

This paper deals with nonparametric estimation of a regression curve, where the estimation method should preserve possible jumps in the curve. At each point x at which one wants to estimate the regression function, the method chooses in an adaptive way among three estimates: a local linear estimate using only datapoints to the left of x, a local linear estimate based on only datapoints to the right of x, and finally a local linear estimate using data in a two-sided neighbourhood around x. The choice among these three estimates is made by looking at differences of the weighted residual mean squares of the three fits. The resulting estimate preserves the jumps well and in addition gives smooth estimates of the continuity parts of the curve. This property of compromise between local smoothing and jump-preserving is what distinguishes our method from most previously proposed methods, that mainly focused on local smoothing and consequently blurred possible jumps, or mainly focused on jump-preserving and hence led to rather noisy estimates in continuity regions of the underlying regression curve. Strong consistency of the estimator is established and its performance is tested via a simulation study. This study also compares the current method with some existing methods. The current method is illustrated in analyzing a real dataset.     

Efficient Quantile Estimation for Functional-Coefficient Partially Linear Regression Models

The quantile estimation methods are proposed for functional-coefficient partially linear regression (FCPLR) model by combining nonparametric and functional-coefficient regression (FCR) model. The local linear scheme and the integrated method are used to obtain local quantile estimators of all unknown functions in the FCPLR model. These resulting estimators are asymptotically normal, but each of them has big variance. To reduce variances of these quantile estimators, the one-step backfitting technique is used to obtain the efficient quantile estimators of all unknown functions, and their asymptotic normalities are derived. Two simulated examples are carried out to illustrate the proposed estimation methodology.     

Comparison of presmoothing methods in kernel density estimation under censoring

The behavior of the presmoothed density estimator is studied when different ways to estimate the conditional probability of uncensoring are used. The Nadaraya–Watson, local linear and local logistic approach are compared via simulations with the classical Kaplan–Meier estimator. While the local logistic presmoothing estimator presents the best performance, the relative benefits of the local linear versus the Nadaraya–Watson estimator depend very much on the shape of some underlying functions.     

Asymptotically optimal estimation in the linear regression problem in the case of violation of some classical assumptions

We consider the problem of estimating the unknown parameters of linear regression in the case when the variances of observations depend on the unknown parameters of the model. A two-step method is suggested for constructing asymptotically linear estimators. Some general sufficient conditions for the asymptotic normality of the estimators are found, and an explicit form is established of the best asymptotically linear estimators. The behavior of the estimators is studied in detail in the case when the parameter of the regression model is one-dimensional.     

Aggregated estimators and empirical complexity for least square regression

Numerous empirical results have shown that combining regression procedures can be a very efficient method. This work provides PAC bounds for the L² generalization error of such methods. The interest of these bounds are twofold.First, it gives for any aggregating procedure a bound for the expected risk depending on the empirical risk and the empirical complexity measured by the Kullback–Leibler divergence between the aggregating distribution and a prior distribution π and by the empirical mean of the variance of the regression functions under the probability .Secondly, by structural risk minimization, we derive an aggregating procedure which takes advantage of the unknown properties of the best mixture : when the best convex combination of d regression functions belongs to the d initial functions (i.e. when combining does not make the bias decrease), the convergence rate is of order (logd)/N. In the worst case, our combining procedure achieves a convergence rate of order which is known to be optimal in a uniform sense when (see [A. Nemirovski, in: Probability Summer School, Saint Flour, 1998; Y. Yang, Aggregating regression procedures for a better performance, 2001]).As in AdaBoost, our aggregating distribution tends to favor functions which disagree with the mixture on mispredicted points. Our algorithm is tested on artificial classification data (which have been also used for testing other boosting methods, such as AdaBoost).