Qualitative and infinitesimal robustness of tail-dependent statistical functionals

The main goal of this article is to introduce a new notion of qualitative robustness that applies also to tail-dependent statistical functionals and that allows us to compare statistical functionals in regards to their degree of robustness. By means of new versions of the celebrated Hampel theorem, we show that this degree of robustness can be characterized in terms of certain continuity properties of the statistical functional. The proofs of these results rely on strong uniform Glivenko-Cantelli theorems in fine topologies, which are of independent interest. We also investigate the sensitivity of tail-dependent statistical functionals w.r.t. infinitesimal contaminations, and we introduce a new notion of infinitesimal robustness. The theoretical results are illustrated by means of several examples including general L- and V-functionals.     

Robustness of reweighted Least Squares Kernel Based Regression

Kernel Based Regression (KBR) minimizes a convex risk over a possibly infinite dimensional reproducing kernel Hilbert space. Recently, it was shown that KBR with a least squares loss function may have some undesirable properties from a robustness point of view: even very small amounts of outliers can dramatically affect the estimates. KBR with other loss functions is more robust, but often gives rise to more complicated computations (e.g. for Huber or logistic losses). In classical statistics robustness is often improved by reweighting the original estimate. In this paper we provide a theoretical framework for reweighted Least Squares KBR (LS-KBR) and analyze its robustness. Some important differences are found with respect to linear regression, indicating that LS-KBR with a bounded kernel is much more suited for reweighting. In two special cases our results can be translated into practical guidelines for a good choice of weights, providing robustness as well as fast convergence. In particular a logistic weight function seems an appropriate choice, not only to downweight outliers, but also to improve performance at heavy tailed distributions. For the latter some heuristic arguments are given comparing concepts from robustness and stability.     

Single-index quantile regression

Nonparametric quantile regression with multivariate covariates is a difficult estimation problem due to the “curse of dimensionality”. To reduce the dimensionality while still retaining the flexibility of a nonparametric model, we propose modeling the conditional quantile by a single-index function , where a univariate link function g₀(⋅) is applied to a linear combination of covariates , often called the single-index. We introduce a practical algorithm where the unknown link function g₀(⋅) is estimated by local linear quantile regression and the parametric index is estimated through linear quantile regression. Large sample properties of estimators are studied, which facilitate further inference. Both the modeling and estimation approaches are demonstrated by simulation studies and real data applications.     

Robust estimation in generalized semiparametric mixed models for longitudinal data

In this paper, we consider robust generalized estimating equations for the analysis of semiparametric generalized partial linear mixed models (GPLMMs) for longitudinal data. We approximate the non-parametric function in the GPLMM by a regression spline, and make use of bounded scores and leverage-based weights in the estimating equation to achieve robustness against outliers and influential data points, respectively. Under some regularity conditions, the asymptotic properties of the robust estimators are investigated. To avoid the computational problems involving high-dimensional integrals in our estimators, we adopt a robust Monte Carlo Newton-Raphson (RMCNR) algorithm for fitting GPLMMs. Small simulations are carried out to study the behavior of the robust estimates in the presence of outliers, and these estimates are also compared to their corresponding non-robust estimates. The proposed robust method is illustrated in the analysis of two real data sets.     

On rates of convergence in functional linear regression

This paper investigates the rate of convergence of estimating the regression weight function in a functional linear regression model. It is assumed that the predictor as well as the weight function are smooth and periodic in the sense that the derivatives are equal at the boundary points. Assuming that the functional data are observed at discrete points with measurement error, the complex Fourier basis is adopted in estimating the true data and the regression weight function based on the penalized least-squares criterion. The rate of convergence is then derived for both estimators. A simulation study is also provided to illustrate the numerical performance of our approach, and to make a comparison with the principal component regression approach.     

On an asymptotically more efficient estimation of the single-index model

In this note, we revisit the single-index model with heteroscedastic error, and recommend an estimating equation method in terms of transferring restricted least squares to unrestricted least squares: the estimator of the index parameter is asymptotically more efficient than existing estimators in the literature in the sense that it is of a smaller limiting variance.     

On a dimension reduction regression with covariate adjustment

In this paper, we consider a semiparametric modeling with multi-indices when neither the response nor the predictors can be directly observed and there are distortions from some multiplicative factors. In contrast to the existing methods in which the response distortion deteriorates estimation efficacy even for a simple linear model, the dimension reduction technique presented in this paper interestingly does not have to account for distortion of the response variable. The observed response can be used directly whether distortion is present or not. The resulting dimension reduction estimators are shown to be consistent and asymptotically normal. The results can be employed to test whether the central dimension reduction subspace has been estimated appropriately and whether the components in the basis directions in the space are significant. Thus, the method provides an alternative for determining the structural dimension of the subspace and for variable selection. A simulation study is carried out to assess the performance of the proposed method. The analysis of a real dataset demonstrates the potential usefulness of distortion removal.     

Influence functions and efficiencies of the canonical correlation and vector estimates based on scatter and shape matrices

In this paper, the influence functions and limiting distributions of the canonical correlations and coefficients based on affine equivariant scatter matrices are developed for elliptically symmetric distributions. General formulas for limiting variances and covariances of the canonical correlations and canonical vectors based on scatter matrices are obtained. Also the use of the so-called shape matrices in canonical analysis is investigated. The scatter and shape matrices based on the affine equivariant Sign Covariance Matrix as well as the Tyler's shape matrix serve as examples. Their finite sample and limiting efficiencies are compared to those of the Minimum Covariance Determinant estimators and S-estimator through theoretical and simulation studies. The theory is illustrated by an example.     

Asymptotic distributions of nonparametric regression estimators for longitudinal or functional data

The estimation of a regression function by kernel method for longitudinal or functional data is considered. In the context of longitudinal data analysis, a random function typically represents a subject that is often observed at a small number of time points, while in the studies of functional data the random realization is usually measured on a dense grid. However, essentially the same methods can be applied to both sampling plans, as well as in a number of settings lying between them. In this paper general results are derived for the asymptotic distributions of real-valued functions with arguments which are functionals formed by weighted averages of longitudinal or functional data. Asymptotic distributions for the estimators of the mean and covariance functions obtained from noisy observations with the presence of within-subject correlation are studied. These asymptotic normality results are comparable to those standard rates obtained from independent data, which is illustrated in a simulation study. Besides, this paper discusses the conditions associated with sampling plans, which are required for the validity of local properties of kernel-based estimators for longitudinal or functional data.     

Semi-varying coefficient models with a diverging number of components

Semiparametric models with both nonparametric and parametric components have become increasingly useful in many scientific fields, due to their appropriate representation of the trade-off between flexibility and efficiency of statistical models. In this paper we focus on semi-varying coefficient models (a.k.a. varying coefficient partially linear models) in a “large n, diverging p” situation, when both the number of parametric and nonparametric components diverges at appropriate rates, and we only consider the case p=o(n). Consistency of the estimator based on B-splines and asymptotic normality of the linear components are established under suitable assumptions. Interestingly (although not surprisingly) our analysis shows that the number of parametric components can diverge at a faster rate than the number of nonparametric components and the divergence rates of the number of the nonparametric components constrain the allowable divergence rates of the parametric components, which is a new phenomenon not established in the existing literature as far as we know. Finally, the finite sample behavior of the estimator is evaluated by some Monte Carlo studies.     

Homomorphisms of direct powers of algebras

We give an example of n-coconnected algebra that is not (n+1)-coconnected, for any n ≥ 2. Received June 8, 2005; accepted in final form August 28, 2005.     

Nonparametric Berkson regression under normal measurement error and bounded design

Regression data often suffer from the so-called Berkson measurement error which contaminates the design variables. Conventional nonparametric approaches to this errors-in-variables problem usually require rather strong conditions on the support of the design density and that of the contaminated regression function, which seem unrealistic in many cases. In the current note, we introduce a novel nonparametric regression estimator, which is able to identify the regression function on the whole real line under normal Berkson error although the location of the design variables is restricted to some bounded interval. The asymptotic properties of this estimator are investigated and some numerical simulations are provided.     

Using bimodal kernel for inference in nonparametric regression with correlated errors

For nonparametric regression model with fixed design, it is well known that obtaining a correct bandwidth is difficult when errors are correlated. Various methods of bandwidth selection have been proposed, but their successful implementation critically depends on a tuning procedure which requires accurate information about error correlation. Unfortunately, such information is usually hard to obtain since errors are not observable. In this article a new bandwidth selector based on the use of a bimodal kernel is proposed and investigated. It is shown that the new bandwidth selector is quite useful for the tuning procedures of various other methods. Furthermore, the proposed bandwidth selector itself proves to be quite effective when the errors are severely correlated.     

Functional semiparametric partially linear model with autoregressive errors

In this paper, we introduce a functional semiparametric model, where a real-valued random variable is explained by the sum of a unknown linear combination of the components of a multivariate random variable and an unknown transformation of a functional random variable. The errors can be autocorrelated. We focus here on the parametric estimation of the coefficients in the linear combination. First, we use a nonparametric kernel method to remove the effect of the functional explanatory variable. Then, we use generalized least squares approach to obtain an estimator of these coefficients. Under some technical assumptions, we prove consistency and asymptotic normality of our estimator. Finally, we present Monte Carlo simulations that illustrate these characteristics.     

Bandwidth selection for a data sharpening estimator in nonparametric regression

This paper is concerned with data-based selection of the bandwidth for a data sharpening estimator in nonparametric regression. Two kinds of bandwidths are considered: a bandwidth vector which has a different bandwidth for each covariate, and a scalar bandwidth that is common for all covariates. A plug-in method is developed and its theoretical performance is fully investigated. The proposed plug-in method works efficiently in our simulation study.     

Nonparametric regression estimation with general parametric error covariance

The asymptotic distribution for the local linear estimator in nonparametric regression models is established under a general parametric error covariance with dependent and heterogeneously distributed regressors. A two-step estimation procedure that incorporates the parametric information in the error covariance matrix is proposed. Sufficient conditions for its asymptotic normality are given and its efficiency relative to the local linear estimator is established. We give examples of how our results are useful in some recently studied regression models. A Monte Carlo study confirms the asymptotic theory predictions and compares our estimator with some recently proposed alternative estimation procedures.     

Local linear regression for functional predictor and scalar response

The aim of this work is to introduce a new nonparametric regression technique in the context of functional covariate and scalar response. We propose a local linear regression estimator and study its asymptotic behaviour. Its finite-sample performance is compared with a Nadayara-Watson type kernel regression estimator and with the linear regression estimator via a Monte Carlo study and the analysis of two real data sets. In all the scenarios considered, the local linear regression estimator performs better than the kernel one, in the sense that the mean squared prediction error is lower.     

Variance component testing in semiparametric mixed models

It is of considerable interest to test for heteroscedasticity in statistical studies. In this paper, we investigate such a problem under the framework of a semiparametric mixed model. A score test is proposed for the hypothesis that all the variance components are zero. We establish the asymptotic property of the test, and examine its performance in a simulation study. The test is illustrated with the analysis of a longitudinal study of measurements of serum creatinine.     

Data sharpening methods in multivariate local quadratic regression

This paper is concerned with the conditional bias and variance of local quadratic regression to the multivariate predictor variables. Data sharpening methods of nonparametric regression were first proposed by Choi, Hall, Roussion. Recently, a data sharpening estimator of local linear regression was discussed by Naito and Yoshizaki. In this paper, to improve mainly the fitting precision, we extend their results on the asymptotic bias and variance. Using the data sharpening estimator of multivariate local quadratic regression, we are able to derive higher fitting precision. In particular, our approach is simple to implement, since it has an explicit form, and is convenient when analyzing the asymptotic conditional bias and variance of the estimator at the interior and boundary points of the support of the density function.     

Simple and efficient improvements of multivariate local linear regression

This paper studies improvements of multivariate local linear regression. Two intuitively appealing variance reduction techniques are proposed. They both yield estimators that retain the same asymptotic conditional bias as the multivariate local linear estimator and have smaller asymptotic conditional variances. The estimators are further examined in aspects of bandwidth selection, asymptotic relative efficiency and implementation. Their asymptotic relative efficiencies with respect to the multivariate local linear estimator are very attractive and increase exponentially as the number of covariates increases. Data-driven bandwidth selection procedures for the new estimators are straightforward given those for local linear regression. Since the proposed estimators each has a simple form, implementation is easy and requires much less or about the same amount of effort. In addition, boundary corrections are automatic as in the usual multivariate local linear regression.