Robust nonparametric regression estimation

In this paper we define a robust conditional location functional without requiring any moment condition. We apply the nonparametric proposals considered by C. Stone (Ann. Statist. 5 (1977), 595–645) to this functional equation in order to obtain strongly consistent, robust nonparametric estimates of the regression function. We give some examples by using nearest neighbor weights or weights based on kernel methods under no assumptions whatsoever on the probability measure of the vector (X,Y). We also derive strong convergence rates and the asymptotic distribution of the proposed estimates.     

Plug-in method for nonparametric regression

The problem of bandwidth selection for non-parametric kernel regression is considered. We will follow the Nadaraya–Watson and local linear estimator especially. The circular design is assumed in this work to avoid the difficulties caused by boundary effects. Most of bandwidth selectors are based on the residual sum of squares (RSS). It is often observed in simulation studies that these selectors are biased toward undersmoothing. This leads to consideration of a procedure which stabilizes the RSS by modifying the periodogram of the observations. As a result of this procedure, we obtain an estimation of unknown parameters of average mean square error function (AMSE). This process is known as a plug-in method. Simulation studies suggest that the plug-in method could have preferable properties to the classical one. Supported by the MSMT: LC 06024.     

Robust regression function estimation

A robust estimator of the regression function is proposed combining kernel methods as introduced for density estimation and robust location estimation techniques. Weak and strong consistency and asymptotic normality are shown under mild conditions on the kernel sequence. The asymptotic variance is a product from a factor depending only on the kernel and a factor similar to the asymptotic variance in robust estimation of location. The estimation is minimax robust in the sense of Huber (1964). Robust estimation of a location parameter. Ann. Math. Statist.33 73–101.     

Bootstrapping the nonparametric ARCH regression model

In this paper we introduce the nonparametric AR(1)–ARCH(1) model and show weak consistency of the Nadaraya–Watson estimators for the model. We propose a residual and a wild bootstrap method and prove weak consistency of the bootstrap estimators.     

Reducing variance in nonparametric surface estimation

We suggest a method for reducing variance in nonparametric surface estimation. The technique is applicable to a wide range of inferential problems, including both density estimation and regression, and to a wide variety of estimator types. It is based on estimating the contours of a surface by minimising deviations of elementary surface estimates along a quadratic curve. Once a contour estimate has been obtained, the final surface estimate is computed by averaging conventional surface estimates along a portion of the contour. Theoretical and numerical properties of the technique are discussed.     

Stahel-Donoho kernel estimation for fixed design nonparametric regression models

This paper reports a robust kernel estimation for fixed design nonparametric regression models. A Stahel-Donoho kernel estimation is introduced, in which the weight functions depend on both the depths of data and the distances between the design points and the estimation points. Based on a local approximation, a computational technique is given to approximate to the incomputable depths of the errors. As a result the new estimator is computationally efficient. The proposed estimator attains a high breakdown point and has perfect asymptotic behaviors such as the asymptotic normality and convergence in the mean squared error. Unlike the depth-weighted estimator for parametric regression models, this depth-weighted nonparametric estimator has a simple variance structure and then we can compare its efficiency with the original one. Some simulations show that the new method can smooth the regression estimation and achieve some desirable balances between robustness and efficiency.     

Data Adaptive Ridging in Local Polynomial Regression

Abstract

When estimating a regression function or its derivatives, local polynomials are an attractive choice due to their flexibility and asymptotic performance. Seifert and Gasser proposed ridging of local polynomials to overcome problems with variance for random design while retaining their advantages. In this article we present a data-independent rule of thumb and a data-adaptive spatial choice of the ridge parameter in local linear regression. In a framework of penalized local least squares regression, the methods are generalized to higher order polynomials, to estimation of derivatives, and to multivariate designs. The main message is that ridging is a powerful tool for improving the performance of local polynomials. A rule of thumb offers drastic improvements; data-adaptive ridging brings further but modest gains in mean square error.     

Efficient nonparametric testing by functional estimation

Many nonparametric tests admit improvement by identifying a functional on a set of probability measures , of which the test statistic is an estimator. We call such a functional a gauge for the problem if it induces the partition of into null and alternative and enjoys certain invariance properties. Two nonparametric testing problems are explored here: a dependency problem and an equidistribution problem. In each a dual smoothing problem is posed and optimally solved in the estimation framework, and a corresponding testing procedure gives a consistency rate improvement over the original test.     

On the estimation of a monotone conditional variance in nonparametric regression

A monotone estimate of the conditional variance function in a heteroscedastic, nonparametric regression model is proposed. The method is based on the application of a kernel density estimate to an unconstrained estimate of the variance function and yields an estimate of the inverse variance function. The final monotone estimate of the variance function is obtained by an inversion of this function. The method is applicable to a broad class of nonparametric estimates of the conditional variance and particularly attractive to users of conventional kernel methods, because it does not require constrained optimization techniques. The approach is also illustrated by means of a simulation study.     

A bias corrected nonparametric regression estimator

In this article, we propose a new method of bias reduction in nonparametric regression estimation. The proposed new estimator has asymptotic bias order h⁴, where h is a smoothing parameter, in contrast to the usual bias order h² for the local linear regression. In addition, the proposed estimator has the same order of the asymptotic variance as the local linear regression. Our proposed method is closely related to the bias reduction method for kernel density estimation proposed by Chung and Lindsay (2011). However, our method is not a direct extension of their density estimate, but a totally new one based on the bias cancelation result of their proof.     

Robust adaptive efficient estimation for semi-Markov nonparametric regression models

Statistical Inference for Stochastic Processes - We consider the nonparametric robust estimation problem for regression models in continuous time with semi-Markov noises. An adaptive model...     

Optimal testing for additivity in multiple nonparametric regression

We consider the problem of testing for additivity in the standard multiple nonparametric regression model. We derive optimal (in the minimax sense) non- adaptive and adaptive hypothesis testing procedures for additivity against the composite nonparametric alternative that the response function involves interactions of second or higher orders separated away from zero in L ²([0, 1] ^d )-norm and also possesses some smoothness properties. In order to shed some light on the theoretical results obtained, we carry out a wide simulation study to examine the finite sample performance of the proposed hypothesis testing procedures and compare them with a series of other tests for additivity available in the literature.     

Variable selection of generalized regression models based on maximum rank correlation

In this paper, we investigate the variable selection problem of the generalized regression models. To estimate the regression parameter, a procedure combining the rank correlation method and the adaptive lasso technique is developed, which is proved to have oracle properties. A modified IMO （iterative marginal optimization） algorithm which directly aims to maximize the penalized rank correlation function is proposed. The effects of the estimating procedure are illustrated by simulation studies.     

A prior near-ignorance Gaussian process model for nonparametric regression

This paper proposes a prior near-ignorance model for regression based on a set of Gaussian Processes (GP). GPs are natural prior distributions for Bayesian regression. They offer a great modeling flexibility and have found widespread application in many regression problems. However, a GP requires the prior elicitation of its mean function, which represents our prior belief about the shape of the regression function, and of the covariance between any two function values.In the absence of prior information, it may be difficult to fully specify these infinite dimensional parameters. In this work, by modeling the prior mean of the GP as a linear combination of a set of basis functions and assuming as prior for the combination coefficients a set of conjugate distributions obtained as limits of truncate exponential priors, we have been able to model prior ignorance about the mean of the GP. The resulting model satisfies translation invariance, learning and, under some constraints, convergence, which are desirable properties for a prior near-ignorance model. Moreover, it is shown in this paper how this model can be extended to allow for a weaker specification of the GP covariance between function values, by letting each basis function to vary in a set of functions.Application to hypothesis testing has shown how the use of this model induces the capability of automatically detecting when a reliable decision cannot be made based on the available data.     

Robust regression credibility: The influence function approach

In classical credibility theory we assume that the vector of claims conditionally on has independent components with identical means. However, this assumption is sometimes unrealistic. To relax this condition Hachemeister (Hachemeister, C.A., 1975. Credibility for regression models with application to trend. In: Kahn, P. (Ed.), Credibility, Theory and Applications. Academic Press, New York) introduced regressors. The presence of large claims can perturb the credibility premium estimation. The lack of robustness of regression credibility estimators, as well as the fairness of tariff evaluation, led to the development of this paper. Our proposal is to apply robust statistics to the regression credibility estimation by using the robust influence function approach of M-estimators.     

Robust and accurate inference for generalized linear models

In the framework of generalized linear models, the nonrobustness of classical estimators and tests for the parameters is a well known problem, and alternative methods have been proposed in the literature. These methods are robust and can cope with deviations from the assumed distribution. However, they are based on first order asymptotic theory, and their accuracy in moderate to small samples is still an open question. In this paper, we propose a test statistic which combines robustness and good accuracy for moderate to small sample sizes. We combine results from Cantoni and Ronchetti [E. Cantoni, E. Ronchetti, Robust inference for generalized linear models, Journal of the American Statistical Association 96 (2001) 1022–1030] and Robinson, Ronchetti and Young [J. Robinson, E. Ronchetti, G.A. Young, Saddlepoint approximations and tests based on multivariate M-estimators, The Annals of Statistics 31 (2003) 1154–1169] to obtain a robust test statistic for hypothesis testing and variable selection, which is asymptotically χ²-distributed as the three classical tests but with a relative error of order O(n⁻¹). This leads to reliable inference in the presence of small deviations from the assumed model distribution, and to accurate testing and variable selection, even in moderate to small samples.     

A smoothing stochastic algorithm for quantile estimation

In this paper, we provide the almost-sure convergence and the asymptotic normality of a smooth version of the Robbins–Monro algorithm for the quantile estimation. A Monte Carlo simulation study shows that our proposed method works well within the framework of a data stream.     

Robust Estimation Using the Huber Function With a Data-Dependent Tuning Constant

Robust estimation often relies on a dispersion function that is more slowly varying at large values than the square function. However, the choice of tuning constant in dispersion functions may impact the estimation efficiency to a great extent. For a given family of dispersion functions such as the Huber family, we suggest obtaining the “best” tuning constant from the data so that the asymptotic efficiency is maximized. This data-driven approach can automatically adjust the value of the tuning constant to provide the necessary resistance against outliers. Simulation studies show that substantial efficiency can be gained by this data-dependent approach compared with the traditional approach in which the tuning constant is fixed. We briefly illustrate the proposed method using two datasets.     

Testing for no effect in nonparametric regression via spline smoothing techniques

We propose three statistics for testing that a predictor variable has no effect on the response variable in regression analysis. The test statistics are integrals of squared derivatives of various orders of a periodic smoothing spline fit to the data. The large sample properties of the test statistics are investigated under the null hypothesis and sequences of local alternatives and a Monte Carlo study is conducted to assess finite sample power properties.     

A robust and efficient estimation method for single-index varying-coefficient models

A new estimation procedure based on modal regression is proposed for single-index varying-coefficient models. The proposed method achieves better robustness and efficiency than that of Xue and Pang (2013). We establish the asymptotic normalities of proposed estimators and evaluate the performance of the proposed method by a numerical simulation.