Applying robust regression to insurance

A statistical procedure is called robust if it is insensitive to the occurence of gross errors in the data. The ordinary least squares regression technique does not satisfy this property, because even a single outlier can totally offset the result. Therefore, the least trimmed squares (LTS) technique is introduced, which can resist the effect of a large percentage of outliers. The latter method is illustrated on data concerning life insurance, pension funds, health insurance, and inflation.     

Quadratic mixed integer programming and support vectors for deleting outliers in robust regression

We consider the problem of deleting bad influential observations (outliers) in linear regression models. The problem is formulated as a Quadratic Mixed Integer Programming (QMIP) problem, where penalty costs for discarding outliers are used into the objective function. The optimum solution defines a robust regression estimator called penalized trimmed squares (PTS). Due to the high computational complexity of the resulting QMIP problem, the proposed robust procedure is computationally suitable for small sample data. The computational performance and the effectiveness of the new procedure are improved significantly by using the idea of ε-Insensitive loss function from support vectors machine regression. Small errors are ignored, and the mathematical formula gains the sparseness property. The good performance of the ε-Insensitive PTS (IPTS) estimator allows identification of multiple outliers avoiding masking or swamping effects. The computational effectiveness and successful outlier detection of the proposed method is demonstrated via simulated experiments. This research has been partially funded by the Greek Ministry of Education under the program Pythagoras II.     

Efficient robust estimation of time-series regression models

The paper studies a new class of robust regression estimators based on the two-step least weighted squares (2S-LWS) estimator which employs data-adaptive weights determined from the empirical distribution or quantile functions of regression residuals obtained from an initial robust fit. Just like many existing two-step robust methods, the proposed 2S-LWS estimator preserves robust properties of the initial robust estimate. However, contrary to the existing methods, the first-order asymptotic behavior of 2S-LWS is fully independent of the initial estimate under mild conditions. We propose data-adaptive weighting schemes that perform well both in the cross-section and time-series data and prove the asymptotic normality and efficiency of the resulting procedure. A simulation study documents these theoretical properties in finite samples.     

A COMPARISON BETWEEN TWO ROBUST REGRESSION ESTIMATORS BY MEANS OF ROBUST COVARIANCES

Two classes of Mallows GM-estimators with invariance are considered in the stochastic linear regression model. Some of their asymptotic properties are described, and the fittedvalue influence and variance components are compared by means of robust covariances,     

过两个样本点的最小一乘回归线

本文用一种选代法，求过两个样本点的最小一乘回归线．     

A study over the general formula of regression sum of squares in multiple linear regression

In this study, in addition to the formula of regression sum of squares (SSR) in linear regression, a general formula of SSR in multiple linear regression is given. The derivations of the formula presented are given step by step. This new formula is proposed for estimation of the SSR in multiple linear regression. By using this formula, the researcher can find easily SSR and so the researcher can compose easily the table of variance analysis to interpret the regression made.     

Deleting Outliers in Robust Regression with Mixed Integer Programming

In robust regression we often have to decide how many are the unusual observations, which should be removed from the sample in order to obtain better fitting for the rest of the observations. Generally, we use the basic principle of LTS, which is to fit the majority of the data, identifying as outliers those points that cause the biggest damage to the robust fit. However, in the LTS regression method the choice of default values for high break down-point affects seriously the efficiency of the estimator. In the proposed approach we introduce penalty cost for discarding an outlier, consequently, the best fit for the majority of the data is obtained by discarding only catastrophic observations. This penalty cost is based on robust design weights and high break down-point residual scale taken from the LTS estimator. The robust estimation is obtained by solving a convex quadratic mixed integer programming problem, where in the objective function the sum of the squared residuals and penalties for discarding observations is minimized. The proposed mathematical programming formula is suitable for small-sample data. Moreover, we conduct a simulation study to compare other robust estimators with our approach in terms of their efficiency and robustness.     

Local linear regression for data with AR errors

In many statistical applications, data are collected over time, and they are likely correlated. In this paper, we investigate how to incorporate the correlation information into the local linear regression. Under the assumption that the error process is an auto-regressive process, a new estimation procedure is proposed for the nonparametric regression by using local linear regression method and the profile least squares techniques. We further propose the SCAD penalized profile least squares method to determine the order of auto-regressive process. Extensive Monte Carlo simulation studies are conducted to examine the finite sample performance of the proposed procedure, and to compare the performance of the proposed procedures with the existing one. From our empirical studies, the newly proposed procedures can dramatically improve the accuracy of naive local linear regression with working-independent error structure. We illustrate the proposed methodology by an analysis of real data set.     

Multiple criteria linear regression

The unknown parameters in multiple linear regression models may be estimated using any one of a number of criteria such as the minimization of the sum of squared errors MSSE, the minimization of the sum of absolute errors MSAE, and the minimization of the maximum absolute error MMAE. At present, the MSSE or the least squares criterion continues to be the most popular. However, at times the choice of a criterion is not clear from statistical, practical or other considerations. Under such circumstances, it may be more appropriate to use multiple criteria rather than a single criterion to estimate the unknown parameters in a multiple linear regression model. We motivate the use of multiple criteria estimation in linear regression models with an example, propose a few models, and outline a solution procedure.     

Central limit theorem of linear regression model under right censorship

In this paper,the estimation of joint dlstribution F(y,z)of(Y,Z)and the estimation in thelinear regression model Y=b'Z+εfor complete data are extended to that of the right censored data.Theregression parameter estimates of b and the variance of ε are weighted least square estimates with randomweights. The central limit theorems of the estimators are obtained under very weak conditions and the derivedasymptotic variance has a very simple form.     

Robust Frequency Estimation Using Elemental Sets

Abstract

The extraction of sinusoidal signals from time-series data is a classic problem of ongoing interest in the statistics and signal processing literatures. Obtaining least squares estimates is difficult because the sum of squares has local minima O(1/n) apart in the frequencies. In practice the frequencies are often estimated using ad hoc and inefficient methods. Problems of data quality have received little attention. An elemental set is a subset of the data containing the minimum number of points such that the unknown parameters in the model can be identified. This article shows that, using a variant of the classical method of Prony, parameter estimates for a sum of sinusoids can be obtained algebraically from an elemental set. Elemental set methods are used to construct finite algorithm estimators that approximately minimize the least squares, least trimmed sum of squares, or least median of squares criteria. The elemental set estimators prove able in simulations to resolve the frequencies to the correct local minima of the objective functions. When used as the first stage of an MM estimator, the constructed estimators based on the trimmed sum of squares and least median of squares criteria produce final estimators which have high breakdown properties and which are simultaneously efficient when no outliers are present. The approach can also be applied to sums of exponentials, and sums of damped sinusoids. The article includes simulations with one and two sinusoids and two data examples.     

Computing S Estimators for Regression and Multivariate Location/Dispersion

Abstract

An improved resampling algorithm for S estimators reduces the number of times the objective function is evaluated and increases the speed of convergence. With this algorithm, S estimates can be computed in less time than least median squares (LMS) for regression and minimum volume ellipsoid (MVE) for location/scatter estimates with the same accuracy. Here accuracy refers to the randomness due to the algorithm. S estimators are also more statistically efficient than the LMS and MVE estimators, that is, they have less variability due to the randomness of the data.     

无界抽样情形下不定核的系数正则化回归

本文讨论样本依赖空间中无界抽样情形下最小二乘损失函数的系数正则化问题. 这里的学习准则与之前再生核Hilbert空间的准则有着本质差异: 核除了满足连续性和有界性之外, 不需要再满足对称性和正定性; 正则化子是函数关于样本展开系数的l²-范数; 样本输出是无界的. 上述差异给误差分析增加了额外难度. 本文的目的是在样本输出不满足一致有界的情形下, 通过l²-经验覆盖数给出误差的集中估计(concentration estimates). 通过引入一个恰当的Hilbert空间以及l²-经验覆盖数的技巧, 得到了与假设空间的容量以及与回归函数的正则性有关的较满意的学习速率.     

The rate of convergence of the least squares estimator in a non-linear regression model with dependent errors

The rate of convergence of the least squares estimator in a non-linear regression model with errors forming either a φ-mixing or strong mixing process is obtained. Strong consistency of the least squares estimator is obtained as a corollary.