Jump-Preserving Regression and Smoothing using Local Linear Fitting: A Compromise

This paper deals with nonparametric estimation of a regression curve, where the estimation method should preserve possible jumps in the curve. At each point x at which one wants to estimate the regression function, the method chooses in an adaptive way among three estimates: a local linear estimate using only datapoints to the left of x, a local linear estimate based on only datapoints to the right of x, and finally a local linear estimate using data in a two-sided neighbourhood around x. The choice among these three estimates is made by looking at differences of the weighted residual mean squares of the three fits. The resulting estimate preserves the jumps well and in addition gives smooth estimates of the continuity parts of the curve. This property of compromise between local smoothing and jump-preserving is what distinguishes our method from most previously proposed methods, that mainly focused on local smoothing and consequently blurred possible jumps, or mainly focused on jump-preserving and hence led to rather noisy estimates in continuity regions of the underlying regression curve. Strong consistency of the estimator is established and its performance is tested via a simulation study. This study also compares the current method with some existing methods. The current method is illustrated in analyzing a real dataset.     

Efficient and fast spline-backfitted kernel smoothing of additive models

A great deal of effort has been devoted to the inference of additive model in the last decade. Among existing procedures, the kernel type are too costly to implement for high dimensions or large sample sizes, while the spline type provide no asymptotic distribution or uniform convergence. We propose a one step backfitting estimator of the component function in an additive regression model, using spline estimators in the first stage followed by kernel/local linear estimators. Under weak conditions, the proposed estimator’s pointwise distribution is asymptotically equivalent to an univariate kernel/local linear estimator, hence the dimension is effectively reduced to one at any point. This dimension reduction holds uniformly over an interval under assumptions of normal errors. Monte Carlo evidence supports the asymptotic results for dimensions ranging from low to very high, and sample sizes ranging from moderate to large. The proposed confidence band is applied to the Boston housing data for linearity diagnosis. Supported in part by NSF awards DMS 0405330, 0706518, BCS 0308420 and SES 0127722.     

Statistical estimation in varying coefficient models with surrogate data and validation sampling

Varying coefficient error-in-covariables models are considered with surrogate data and validation sampling. Without specifying any error structure equation, two estimators for the coefficient function vector are suggested by using the local linear kernel smoothing technique. The proposed estimators are proved to be asymptotically normal. A bootstrap procedure is suggested to estimate the asymptotic variances. The data-driven bandwidth selection method is discussed. A simulation study is conducted to evaluate the proposed estimating methods.     

Local Linear Estimation of Jump-Diffusion Models by Using Asymmetric Kernels

This article addresses the problem of nonparametric estimation of the first and second infinitesimal moments by using the local linear method of the underlying jump-diffusion models. The motivation behind the study is to use the asymmetric kernels instead of standard kernel smoothing. The basic idea relies on replacing the symmetric kernel by asymmetric kernel and provides a new way of obtaining the nonparametric estimation for jump-diffusion models. We prove that the estimators based on the local linear method for jump-diffusion models are consistent and asymptotically follow normal distribution under the condition of recurrence and stationarity.     

Robustness of one-sided cross-validation to autocorrelation

The effects of moderate levels of serial correlation on one-sided and ordinary cross-validation in the context of local linear and kernel smoothing is investigated. It is shown both theoretically and by simulation that one-sided cross-validation is much less adversely affected by correlation than is ordinary cross-validation. The former method is a reliable means of window width selection in the presence of moderate levels of serial correlation, while the latter is not. It is also shown that ordinary cross-validation is less robust to correlation when applied to Gasser-Müller kernel estimators than to local linear ones.     

Efficient estimation of functional-coefficient regression models with different smoothing variables

In this article,a procedure for estimating the coefficient functions on the functional-coefficient regression models with different smoothing variables in different coefficient functions is defined.First step,by the local linear technique and the averaged method,the initial estimates of the coefficient functions are given.Second step,based on the initial estimates,the efficient estimates of the coefficient functions are proposed by a one-step back-fitting procedure.The efficient estimators share the same asymptotic normalities as the local linear estimators for the functional-coefficient models with a single smoothing variable in different functions.Two simulated examples show that the procedure is effective.     

Data-Adaptive Estimation of Time-Varying Spectral Densities

This article introduces a data-adaptive nonparametric approach for the estimation of time-varying spectral densities from nonstationary time series. Time-varying spectral densities are commonly estimated by local kernel smoothing. The performance of these nonparametric estimators, however, depends crucially on the smoothing bandwidths that need to be specified in both time and frequency direction. As an alternative and extension to traditional bandwidth selection methods, we propose an iterative algorithm for constructing localized smoothing kernels data-adaptively. The main idea, inspired by the concept of propagation-separation, is to determine for a point in the time-frequency plane the largest local vicinity over which smoothing is justified by the data. By shaping the smoothing kernels nonparametrically, our method not only avoids the problem of bandwidth selection in the strict sense but also becomes more flexible. It not only adapts to changing curvature in smoothly varying spectra but also adjusts for structural breaks in the time-varying spectrum. Supplementary materials, including the R package tvspecAdapt containing an implementation of the routine, are available online.     

纵向数据边际模型非参数光滑方法的比较

对于纵向数据边际模型的均值函数, 有很多非参数估计方法, 其中回归样条, 光滑样条, 似乎不相关(SUR)核估计等方法在工作协方差阵正确指定时具有最小的渐近方差. 回归样条的渐近偏差与工作协方差阵无关, 而SUR核估计和光滑样条估计的渐近偏差却依赖于工作协方差阵. 本文主要研究了回归样条, 光滑样条和SUR核估计的效率问题. 通过模拟比较发现回归样条估计的表现比较稳定, 在大多数情况下比光滑样条估计和SUR核估计的效率高.     

Asymptotically Efficient Sequential Kernel Estimates of the Drift Coefficient in Ergodic Diffusion Processes

A sequential asymptotically efficient procedure is constructed for estimating the drift coefficient at a given state point in ergodic diffusion processes. Sequential kernel estimators are used. The optimal convergence rate with the sharp constant is given for a local minimax risk.     

Fast Algorithms for Nonparametric Curve Estimation

Abstract

Naive implementations of local polynomial fits and kernel estimators require almost O(n ²) operations. In this article two fast O(n) algorithms for nonparametric local polynomial fitting are presented. They are based on updating normal equations. Numerical stability is guaranteed by controlling ill-conditioned situations for small bandwidths and data-tuned restarting of the updating procedure. Restarting at every output point results in a moderately fast but highly stable O(n ^7/5) algorithm. Applicability of algorithms is evaluated for estimation of regression curves and their derivatives. The idea is also applied to kernel estimators of regression curves and densities.     

不同自变量的变系数模型的估计

讨论具有不同自变量的变系数模型的函数系数的估计及其大样本性质。使用局部线性方法和积分方法,得到函数系数的积分估计;由于该估计有较大的方差,进一步使用回切法改进这一估计,获得了函数系数的改进估计;同时,研究了改进估计的渐近正态性。最后,用模拟例子说明提出的估计方法是有效的。     

Locally adaptive hazard smoothing

Summary We consider nonparametric estimation of hazard functions and their derivatives under random censorship, based on kernel smoothing of the Nelson (1972) estimator. One critically important ingredient for smoothing methods is the choice of an appropriate bandwidth. Since local variance of these estimates depends on the point where the hazard function is estimated and the bandwidth determines the trade-off between local variance and local bias, data-based local bandwidth choice is proposed. A general principle for obtaining asymptotically efficient data-based local bandwiths, is obtained by means of weak convergence of a local bandwidth process to a Gaussian limit process. Several specific asymptotically efficient bandwidth estimators are discussed. We propose in particular an, asymptotically efficient method derived from direct pilot estimators of the hazard function and of the local mean squared error. This bandwidth choice method has practical advantages and is also of interest in the uncensored case as well as for density estimation.Research supported by UC Davis Faculty Research Grant and by Air Force grant AFOSR-89-0386Research supported by Air Force grant AFOSR-89-0386     

Local Polynomial Regression: Optimal Kernels and Asymptotic Minimax Efficiency

We consider local polynomial fitting for estimating a regression function and its derivatives nonparametrically. This method possesses many nice features, among which automatic adaptation to the boundary and adaptation to various designs. A first contribution of this paper is the derivation of an optimal kernel for local polynomial regression, revealing that there is a universal optimal weighting scheme. Fan (1993, Ann. Statist., 21, 196-216) showed that the univariate local linear regression estimator is the best linear smoother, meaning that it attains the asymptotic linear minimax risk. Moreover, this smoother has high minimax risk. We show that this property also holds for the multivariate local linear regression estimator. In the univariate case we investigate minimax efficiency of local polynomial regression estimators, and find that the asymptotic minimax efficiency for commonly-used orders of fit is 100% among the class of all linear smoothers. Further, we quantify the loss in efficiency when going beyond this class.     

Detection of a change point based on local-likelihood

In this paper, we consider the regression function or its νth derivative in generalized linear models which may have a change/discontinuity point at an unknown location. The location and its jump size are estimated with the local polynomial fits based on one-sided kernel weighted local-likelihood functions. Asymptotic distributions of the proposed estimators of location and jump size are established. The finite-sample performances of the proposed estimators with practical aspects are illustrated by simulated and beetle mortality examples.     

Numerical discretization-based kernel type estimation methods for ordinary differential equation models

We consider the problem of parameter estimation in both linear and nonlinear ordinary differential equation(ODE) models. Nonlinear ODE models are widely used in applications. But their analytic solutions are usually not available. Thus regular methods usually depend on repetitive use of numerical solutions which bring huge computational cost. We proposed a new two-stage approach which includes a smoothing method(kernel smoothing or local polynomial fitting) in the first stage, and a numerical discretization method(Eulers discretization method, the trapezoidal discretization method,or the Runge–Kutta discretization method) in the second stage. Through numerical simulations, we find the proposed method gains a proper balance between estimation accuracy and computational cost.Asymptotic properties are also presented, which show the consistency and asymptotic normality of estimators under some mild conditions. The proposed method is compared to existing methods in term of accuracy and computational cost. The simulation results show that the estimators with local linear smoothing in the first stage and trapezoidal discretization in the second stage have the lowest average relative errors. We apply the proposed method to HIV dynamics data to illustrate the practicability of the estimator.     

Multivariate Locally Weighted Polynomial Fitting and Partial Derivative Estimation

Nonparametric regression estimator based on locally weighted least squares fitting has been studied by Fan and Ruppert and Wand. The latter paper also studies, in the univariate case, nonparametric derivative estimators given by a locally weighted polynomial fitting. Compared with traditional kernel estimators, these estimators are often of simpler form and possess some better properties. In this paper, we develop current work on locally weighted regression and generalize locally weighted polynomial fitting to the estimation of partial derivatives in a multivariate regression context. Specifically, for both the regression and partial derivative estimators we prove joint asymptotic normality and derive explicit asymptotic expansions for their conditional bias and conditional convariance matrix (given observations of predictor variables) in each of the two important cases of local linear fit and local quadratic fit.     

Likelihood estimation of the extremal index

The article develops the approach of Ferro and Segers (J.R. Stat. Soc., Ser. B 65:545, 2003) to the estimation of the extremal index, and proposes the use of a new variable decreasing the bias of the likelihood based on the point process character of the exceedances. Two estimators are discussed: a maximum likelihood estimator and an iterative least squares estimator based on the normalized gaps between clusters. The first provides a flexible tool for use with smoothing methods. A diagnostic is given for condition , under which maximum likelihood is valid. The performance of the new estimators were tested by extensive simulations. An application to the Central England temperature series demonstrates the use of the maximum likelihood estimator together with smoothing methods.      

Estimating a density under pointwise constraints on the derivatives

Suppose we want to estimate a density at a point where we know the values of its first or higher order derivatives. In this case a given kernel estimator of the density can be modified by adding appropriately weighted kernel estimators of these derivatives. We give conditions under which the modified estimators are asymptotically normal. We also determine the optimal weights. When the highest derivative is known to vanish at a point, then the bias is asymptotically negligible at that point and the asymptotic variance of the kernel estimator can be made arbitrarily small by choosing a large bandwidth.     

Reduced‐bias kernel estimators of a positive extreme value index

In this paper, we deal with the semi‐parametric estimation of the extreme value index, an important parameter in extreme value analysis. It is well known that many classic estimators, such as the Hill estimator, reveal a strong bias. This problem motivated the study of two classes of kernel estimators. Those classes generalize the classical Hill estimator and have a tuning parameter that enables us to modify the asymptotic mean squared error and eventually to improve their efficiency. Since the improvement in efficiency is not very expressive, we also study new reduced bias estimators based on the two classes of kernel statistics. Under suitable conditions, we prove their asymptotic normality. Moreover, an asymptotic comparison, at optimal levels, shows that the new classes of reduced bias estimators are more efficient than other reduced bias estimator from the literature. An illustration of the finite sample behaviour of the kernel reduced‐bias estimators is also provided through the analysis of a data set in the field of insurance.     

左截断相依数据下条件分位数的双核局部线性估计

本文在左截断相依数据下,利用局部线性估计的方法,先提出了条件分布函数的双核估计;然后利用该估计导出了条件分位数的双核局部线性估计,并建立了这些估计的渐近正态性结果;最后,通过模拟显示该估计在偏移和边界点调节上要比一般的核估计更好.