Robustness of Stein-type estimators under a non-scalar error covariance structure

The Stein-rule (SR) and positive-part Stein-rule (PSR) estimators are two popular shrinkage techniques used in linear regression, yet very little is known about the robustness of these estimators to the disturbances’ deviation from the white noise assumption. Recent studies have shown that the OLS estimator is quite robust, but whether this is so for the SR and PSR estimators is less clear as these estimators also depend on the F statistic which is highly susceptible to covariance misspecification. This study attempts to evaluate the effects of misspecifying the disturbances as white noise on the SR and PSR estimators by a sensitivity analysis. Sensitivity statistics of the SR and PSR estimators are derived and their properties are analyzed. We find that the sensitivity statistics of these estimators exhibit very similar properties and both estimators are extremely robust to MA(1) disturbances and reasonably robust to AR(1) disturbances except for the cases of severe autocorrelation. The results are useful in light of the rising interest of the SR and PSR techniques in the applied literature.     

Temporal hierarchies with autocorrelation for load forecasting

We propose four different estimators that take into account the autocorrelation structure when reconciling forecasts in a temporal hierarchy. Combining forecasts from multiple temporal aggregation levels exploits information differences and mitigates model uncertainty, while reconciliation ensures a unified prediction that supports aligned decisions at different horizons. In previous studies, weights assigned to the forecasts were given by the structure of the hierarchy or the forecast error variances without considering potential autocorrelation in the forecast errors. Our first estimator considers the autocovariance matrix within each aggregation level. Since this can be difficult to estimate, we propose a second estimator that blends autocorrelation and variance information, but only requires estimation of the first-order autocorrelation coefficient at each aggregation level. Our third and fourth estimators facilitate information sharing between aggregation levels using robust estimates of the cross-correlation matrix and its inverse. We compare the proposed estimators in a simulation study and demonstrate their usefulness through an application to short-term electricity load forecasting in four price areas in Sweden. We find that by taking account of auto- and cross-covariances when reconciling forecasts, accuracy can be significantly improved uniformly across all frequencies and areas.     

Lower Rank Approximation of Matrices Based on Fast and Robust Alternating Regression

We introduce fast and robust algorithms for lower rank approximation to given matrices based on robust alternating regression. The alternating least squares regression, also called criss-cross regression, was used for lower rank approximation of matrices, but it lacks robustness against outliers in these matrices. We use robust regression estimators and address some of the complications arising from this approach. We find it helpful to use high breakdown estimators in the initial iterations, followed by M estimators with monotone score functions in later iterations towards convergence. In addition to robustness, the computational speed is another important consideration in the development of our proposed algorithm, because alternating robust regression can be computationally intensive for large matrices. Based on a mix of the least trimmed squares (LTS) and Huber's M estimators, we demonstrate that fast and robust lower rank approximations are possible for modestly large matrices.     

Robust weighted orthogonal regression in the errors-in-variables model

This paper focuses on robust estimation in the structural errors-in-variables (EV) model. A new class of robust estimators, called weighted orthogonal regression estimators, is introduced. Robust estimators of the parameters of the EV model are simply derived from robust estimators of multivariate location and scatter such as the M-estimators, the S-estimators and the MCD estimator. The influence functions of the proposed estimators are calculated and shown to be bounded. Moreover, we derive the asymptotic distributions of the estimators and illustrate the results on simulated examples and on a real-data set.     

Multivariate generalized S-estimators

In this paper we introduce generalized S-estimators for the multivariate regression model. This class of estimators combines high robustness and high efficiency. They are defined by minimizing the determinant of a robust estimator of the scatter matrix of differences of residuals. In the special case of a multivariate location model, the generalized S-estimator has the important independence property, and can be used for high breakdown estimation in independent component analysis. Robustness properties of the estimators are investigated by deriving their breakdown point and the influence function. We also study the efficiency of the estimators, both asymptotically and at finite samples. To obtain inference for the regression parameters, we discuss the fast and robust bootstrap for multivariate generalized S-estimators. The method is illustrated on a real data example.     

Preliminary test and Stein estimations in simultaneous linear equations

In modeling of an economic system, there may occur some stochastic constraints, that can cause some changes in the estimators and their respective behaviors. In this approach we formulate the simultaneous equation models into the problem of estimating the regression parameters of a multiple regression model, under elliptical errors. We define five different sorts of estimators for the vector-parameter. Their exact risk expressions are also derived under the balanced loss function. Comparisons are then made to clarify the performance of the proposed estimators. It is shown that the shrinkage factor of the Stein estimator is robust with respect to departures from normality assumption.     

Mallows’ quasi-likelihood estimation for log-linear Poisson autoregressions

We consider the problems of robust estimation and testing for a log-linear model with feedback for the analysis of count time series. We study inference for contaminated data with transient shifts, level shifts and additive outliers. It turns out that the case of additive outliers deserves special attention. We propose a robust method for estimating the regression coefficients in the presence of interventions. The resulting robust estimators are asymptotically normally distributed under some regularity conditions. A robust score type test statistic is also examined. The methodology is applied to real and simulated data.     

Robust estimation in generalized semiparametric mixed models for longitudinal data

In this paper, we consider robust generalized estimating equations for the analysis of semiparametric generalized partial linear mixed models (GPLMMs) for longitudinal data. We approximate the non-parametric function in the GPLMM by a regression spline, and make use of bounded scores and leverage-based weights in the estimating equation to achieve robustness against outliers and influential data points, respectively. Under some regularity conditions, the asymptotic properties of the robust estimators are investigated. To avoid the computational problems involving high-dimensional integrals in our estimators, we adopt a robust Monte Carlo Newton-Raphson (RMCNR) algorithm for fitting GPLMMs. Small simulations are carried out to study the behavior of the robust estimates in the presence of outliers, and these estimates are also compared to their corresponding non-robust estimates. The proposed robust method is illustrated in the analysis of two real data sets.     

Robust Estimation in Partial Linear Mixed Model for Longitudinal Data

In this article, robust generalized estimating equation for the analysis of partial linear mixed model for longitudinal data is used. The authors approximate the nonparametric function by a regression spline. Under some regular conditions, the asymptotic properties of the estimators are obtained. To avoid the computation of high-dimensional integral, a robust Monte Carlo Newton-Raphson algorithm is used. Some simulations are carried out to study the performance of the proposed robust estimators. In addition, the authors also study the robustness and the efficiency of the proposed estimators by simulation. Finally, two real longitudinal data sets are analyzed.     

Some closed form robust moment‐based estimators for the MEM(1,1)

In this paper, we extend the closed form moment estimator (ordinary MCFE) for the autoregressive conditional duration model given by Lu et al (2016) and propose some closed form robust moment‐based estimators for the multiplicative error model to deal with the additive and innovational outliers. The robustification of the closed form estimator is done by replacing the sample mean and sample autocorrelation with some robust estimators. These estimators are more robust than the quasi‐maximum likelihood estimator (QMLE) often used to estimate this model, and they are easy to implement and do not require the use of any numerical optimization procedure and the choice of initial value. The performance of our proposal in estimating the parameters and forecasting conditional mean μ_t of the MEM(1,1) process is compared with the proposals existing in the literature via Monte Carlo experiments, and the results of these experiments show that our proposal outperforms the ordinary MCFE, QMLE, and least absolute deviation estimator in the presence of outliers in general. Finally, we fit the price durations of IBM stock with the robust closed form estimators and the benchmarks and analyze their performances in estimating model parameters and forecasting the irregularly spaced intraday Value at Risk.     

Robust goodness-of-fit tests for AR(p) models based onL 1-norm fitting

A robustified residual autocorrelation is defined based onL ₁-regression. Under very general conditions, the asymptotic distribution of the robust residual autocorrelation is obtained. A robustified portmanteau statistic is then constructed which can be used in checking the goodness-of-fit of AR(p) models when usingL ₁-norm fitting. Empirical results show thatL ₁-norm estimators and the proposed portmanteau statistic are robust against outliers, error distributions, and accuracy for a given finite sample. Project supported by the Foundation of State Educational Commission and a research grant from the Doctoral Program Foundation of China (#97000139).     

Penalized Likelihood-type Estimators for Generalized Nonparametric Regression

We consider the asymptotic analysis of penalized likelihood type estimators for generalized nonparametric regression problems in which the target parameter is a vector-valued function defined in terms of the conditional distribution of a response given a set of covariates. A variety of examples including ones related to generalized linear models and robust smoothing are covered by the theory. Linear approximations to the estimator are constructed using Taylor expansions in Hilbert spaces. An application which is treated is upper bounds on rates of convergence for the penalized likelihood-type estimators.     

Efficient robust estimation of time-series regression models

The paper studies a new class of robust regression estimators based on the two-step least weighted squares (2S-LWS) estimator which employs data-adaptive weights determined from the empirical distribution or quantile functions of regression residuals obtained from an initial robust fit. Just like many existing two-step robust methods, the proposed 2S-LWS estimator preserves robust properties of the initial robust estimate. However, contrary to the existing methods, the first-order asymptotic behavior of 2S-LWS is fully independent of the initial estimate under mild conditions. We propose data-adaptive weighting schemes that perform well both in the cross-section and time-series data and prove the asymptotic normality and efficiency of the resulting procedure. A simulation study documents these theoretical properties in finite samples.     

Robust Depth-Weighted Wavelet for Nonparametric Regression Models

In the nonparametric regression models, the original regression estimators including kernel estimator, Fourier series estimator and wavelet estimator are always constructed by the weighted sum of data, and the weights depend only on the distance between the design points and estimation points. As a result these estimators are not robust to the perturbations in data. In order to avoid this problem, a new nonparametric regression model, called the depth-weighted regression model, is introduced and then the depth-weighted wavelet estimation is defined. The new estimation is robust to the perturbations in data, which attains very high breakdown value close to 1/2. On the other hand, some asymptotic behaviours such as asymptotic normality are obtained. Some simulations illustrate that the proposed wavelet estimator is more robust than the original wavelet estimator and, as a price to pay for the robustness, the new method is slightly less efficient than the original method.     

Stein estimation under elliptical distributions

In a subclass of elliptical distributions, Stein estimators are robust in estimating the mean vector and the regression parameters in a linear regression model. Unbiased estimates of bias and risk are also given for the regression model.     

对稳健回归尺度参数估计的一种改进

常对线性回归模型的稳健 M估计中 ,尺度参数使用绝对离差中位数 MAD.将 Rousseeuw等人对单变量尺度参数的一种稳健估计 Sn引入到回归问题中 ,讨论了此估计的一些优良性质 ,并通过一个小规模的模拟研究 ,说明使用 Sn比使用 MAD做尺度参数将会较大地提高回归估计的估计效率 .     

纵向数据的有效秩推断基于修正的Cholesky分解

本文基于修正的Cholesky分解提出新的方法估计纵向秩回归的组内协方差矩阵,进而提出新的无偏估计函数改善不平衡纵向数据的估计效率.在一些正则条件下,建立了所提估计的渐近正态性.进一步,提出稳健的秩得分检验统计量对回归系数做假设检验.模拟研究和实证分析表明所提方法能够获得高度有效的估计以及所提检验方法比存在的方法更好.     

Robust Nonparametric Function Estimation for Errors-in-variables Models

This paper discusses robust nonparametric estimators of location regression function for errorsin-variables model with de-convolution kernel. The local constant smoother is used for the estimation of the nonparametric function, and the local linear smoother is proposed to deal with the boundary problem, as well as to improve the local constant smoother. We establish the asymptotic properties of the estimator, the influence function of the statistical functional and the breakdown point. A simulat...     

半参数回归模型的稳健补偿最小二乘估计

本文研究了一类半参数回归模型，利用稳健补偿最小二乘估计法，得到了稳健补偿最小二乘估计量，以及它们的影响函数及渐近方差一协方差，对结果的分析表明了该法优于补偿最小二乘法，而且具有稳定性．     

基于相依函数型数据非参数回归函数的稳健核估计

本文研究了基于相依函数型数据非参数回归函数的核估计.利用稳健的方法,在一定条件下获得了与i.i.d.场合下类似的估计量的几乎完全收敛速度,推广了现有文献中的相关结论.