The use of relative residues in non-linear regression

A non-linear curve-fitting model is presented which minimizes the sum of squares of relative residues, and expressions are derived for the fit parameters and their respective errors. A detailed comparison is made between the new general relative least squares model (GRLS) and other non-linear regression models available in the literature, using two sets of data representing fluid mechanics problems encountered in many engineering applications. The results showed that GRLS was the best model for fitting non-linear functions in the case of experimental data spanning several orders of magnitude, indicating its potential as a tool for data analysis.     

The use of relative residues in fitting experimental data: an example from fluid mechanics

A curve fitting model is presented which minimizes the sum of squares of relative residues and expressions for the fit coefficients and their respective errors are derived. The new model is compared to the normal least squares model, using as an example the Reynolds number-drag coefficient data for a sphere. The results show that the best fit was obtained with the new model, indicating it may provide a useful tool for data analysis.     

On the solution of the errors in variables problem using thel 1 norm

A fundamental problem in data analysis is that of fitting a given model to observed data. It is commonly assumed that only the dependent variable values are in error, and the least squares criterion is often used to fit the model. When significant errors occur in all the variables, then an alternative approach which is frequently suggested for this errors in variables problem is to minimize the sum of squared orthogonal distances between each data point and the curve described by the model equation. It has long been recognized that the use of least squares is not always satisfactory, and thel ₁ criterion is often superior when estimating the true form of data which contain some very inaccurate observations. In this paper the measure of goodness of fit is taken to be thel ₁ norm of the errors. A Levenberg-Marquardt method is proposed, and the main objective is to take full advantage of the structure of the subproblems so that they can be solved efficiently.     

Interval estimation for fitting straight line when both variables are subject to error

When both variables are subject to error in regression model, the least squares estimators are biased and inconsistent. The measurement error model is more appropriate to fit the data. This study focuses on the problem to construct interval estimation for fitting straight line in linear measurement error model when one of the error variances is known. We use the concepts of generalized pivotal quantity and construct the confidence interval for the slope because no pivot is available in this case. We compare the existing confidence intervals in terms of coverage probability and expected length via simulation studies. A real data example is also analyzed.     

Orthogonal regression: a teaching perspective

A well-known approach to linear least squares regression is that which involves minimizing the sum of squared orthogonal projections of data points onto the best fit line. This form of regression is known as orthogonal regression, and the linear model that it yields is known as the major axis. A similar method, reduced major axis regression, is predicated on minimizing the total sum of triangular areas formed between data points and the best fit line. Either of these methods is appropriately applied when both x and y are measured, a typical case in the natural sciences. In comparison to classical linear regression, equation derivation for the slope of the major axis and reduced major axis lines is a nontrivial process. For this reason, derivations are presented herein drawing from previous literature with as few steps as possible to enable an easily accessible understanding. Application to eruption data for Old Faithful geyser, Yellowstone National Park, Wyoming and Montana, USA enables a teaching opportunity for choice of model.     

剔除相关性的最小二乘研究

本文提出了一种新的回归模型,剔除相关性的最小二乘,它有效的克服了变量间的相关性,兼顾到变量的筛选。并与最小二乘、向后删除变量法、偏最小二乘比较分析。发现剔除相关性的最小二乘能很好的处理自变量间多重相关性,对变量进行有效的筛选,克服了回归系数反常的现象。     

Least squares estimation of linear regression models for convex compact random sets

Simple and multiple linear regression models are considered between variables whose “values” are convex compact random sets in ${\mathbb{R}^p}$ , (that is, hypercubes, spheres, and so on). We analyze such models within a set-arithmetic approach. Contrary to what happens for random variables, the least squares optimal solutions for the basic affine transformation model do not produce suitable estimates for the linear regression model. First, we derive least squares estimators for the simple linear regression model and examine them from a theoretical perspective. Moreover, the multiple linear regression model is dealt with and a stepwise algorithm is developed in order to find the estimates in this case. The particular problem of the linear regression with interval-valued data is also considered and illustrated by means of a real-life example.     

基于泛岭估计对岭估计过度压缩的改进方法

岭估计是解决多元线性回归多重共线性问题的有效方法,是有偏的压缩估计。与普通最小二乘估计相比,岭估计可以降低参数估计的均方误差,但是却增大残差平方和,拟合效果变差。本文提出一种基于泛岭估计对岭估计过度压缩的改进方法,可以改进岭估计的拟合效果,减小岭估计残差平方和的增加幅度。     

带模糊回归参数的线性回归模型

本文讨论了数值输入模糊数输出的观测数据的线性最小二乘拟合问题，建立了数值空间到模糊数空间的带模糊回归参数的线性回归模型，证明了模型解的存在性和唯一性，并得到了解的表达式。本模型应用简便，具有实用价值。     

Derivative-free optimization and neural networks for robust regression

Large outliers break down linear and nonlinear regression models. Robust regression methods allow one to filter out the outliers when building a model. By replacing the traditional least squares criterion with the least trimmed squares (LTS) criterion, in which half of data is treated as potential outliers, one can fit accurate regression models to strongly contaminated data. High-breakdown methods have become very well established in linear regression, but have started being applied for non-linear regression only recently. In this work, we examine the problem of fitting artificial neural networks (ANNs) to contaminated data using LTS criterion. We introduce a penalized LTS criterion which prevents unnecessary removal of valid data. Training of ANNs leads to a challenging non-smooth global optimization problem. We compare the efficiency of several derivative-free optimization methods in solving it, and show that our approach identifies the outliers correctly when ANNs are used for nonlinear regression.     

Design of an S Function for Robust Regression Using Iteratively Reweighted Least Squares

Abstract

We develop a set of 5 functions for robust regression using the technique of iteratively reweighted least squares (IRLS). Together with a set of weight functions, function rreg is simple to understand and provides great flexibility for IRLS methods. This article focuses on the programming strategies adopted to achieve the twin goals of power and simplicity.     

Neural networks approach for determining total claim amounts in insurance

In this study, we present an approach based on neural networks, as an alternative to the ordinary least squares method, to describe the relation between the dependent and independent variables. It has been suggested to construct a model to describe the relation between dependent and independent variables as an alternative to the ordinary least squares method. A new model, which contains the month and number of payments, is proposed based on real data to determine total claim amounts in insurance as an alternative to the model suggested by Rousseeuw et al. (1984) [Rousseeuw, P., Daniels, B., Leroy, A., 1984. Applying robust regression to insurance. Insurance: Math. Econom. 3, 67–72] in view of an insurer.     

An efficient generalized least squares algorithm for periodic trended regression with autoregressive errors

Time series data with periodic trends like daily temperatures or sales of seasonal products can be seen in periods fluctuating between highs and lows throughout the year. Generalized least squares estimators are often computed for such time series data as these estimators have minimum variance among all linear unbiased estimators. However, the generalized least squares solution can require extremely demanding computation when the data is large. This paper studies an efficient algorithm for generalized least squares estimation in periodic trended regression with autoregressive errors. We develop an algorithm that can substantially simplify generalized least squares computation by manipulating large sets of data into smaller sets. This is accomplished by coining a structured matrix for dimension reduction. Simulations show that the new computation methods using our algorithm can drastically reduce computing time. Our algorithm can be easily adapted to big data that show periodic trends often pertinent to economics, environmental studies, and engineering practices.     

Truncated Estimator of Asymptotic Covariance Matrix in Partially Linear Models with Heteroscedastic Errors

A partially linear regression model with heteroscedastic and/or serially correlated errors is studied here. It is well known that in order to apply the semiparametric least squares estimation （SLSE） to make statistical inference a consistent estimator of the asymptotic covariance matrix is needed. The traditional residual-based estimator of the asymptotic covariance matrix is not consistent when the errors are heteroscedastic and/or serially correlated. In this paper we propose a new estimator by truncating, which is an extension of the procedure in White. This estimator is shown to be consistent when the truncating parameter converges to infinity with some rate.     

Multivariate Locally Weighted Polynomial Fitting and Partial Derivative Estimation

Nonparametric regression estimator based on locally weighted least squares fitting has been studied by Fan and Ruppert and Wand. The latter paper also studies, in the univariate case, nonparametric derivative estimators given by a locally weighted polynomial fitting. Compared with traditional kernel estimators, these estimators are often of simpler form and possess some better properties. In this paper, we develop current work on locally weighted regression and generalize locally weighted polynomial fitting to the estimation of partial derivatives in a multivariate regression context. Specifically, for both the regression and partial derivative estimators we prove joint asymptotic normality and derive explicit asymptotic expansions for their conditional bias and conditional convariance matrix (given observations of predictor variables) in each of the two important cases of local linear fit and local quadratic fit.     

基于半参数可加自回归模型的黄金价格实证分析

大量实证研究表明,半参数自回归模型较传统的线性回归而言,能更好的拟合实际数据。本文构造了一类半参数可加自回归模型,基于条件最小二乘方法及核估计方法给出了估计模型参数和未知函数的迭代算法,讨论了估计量的渐近性质。通过数值模拟验证了估计的效果。并将模型应用于黄金价格数据的实证分析之中。实证分析结果表明,我们对现有模型的改进是必要的。     

Two optimization problems in linear regression with interval data

We consider a linear regression model where neither regressors nor the dependent variable is observable; only intervals are available which are assumed to cover the unobservable data points. Our task is to compute tight bounds for the residual errors of minimum-norm estimators of regression parameters with various norms (corresponding to least absolute deviations (LAD), ordinary least squares (OLS), generalized least squares (GLS) and Chebyshev approximation). The computation of the error bounds can be formulated as a pair of max–min and min–min box-constrained optimization problems. We give a detailed complexity-theoretic analysis of them. First, we prove that they are NP-hard in general. Then, further analysis explains the sources of NP-hardness. We investigate three restrictions when the problem is solvable in polynomial time: the case when the parameter space is known apriori to be restricted into a particular orthant, the case when the regression model has a fixed number of regression parameters, and the case when only the dependent variable is observed with errors. We propose a method, called orthant decomposition of the parameter space, which is the main tool for obtaining polynomial-time computability results.     

基于errors-in-variables的预测模型及其应用

预测是统计学实际应用的一个主要方面,多元线性回归预测是一种很好的方法,广泛地应用在各种实际领域,但其局限性及不足也是明显的。本文以一种新的观点认识数据,即认为变量的观测里均含有误差,同时认为不应删除经慎重选择进来的解释变量。为此,本文提出了一种新的多元预测方法———多元线性EIV预测。本文还考虑了新预测模型的一个实例应用,并从相对偏差上与多元回归预测进行了比较,从而揭示了多元线性EIV预测的先进性及较好的预测精度。     

Local Weighted Composite Quantile Estimating for Varying Coefficient Models

A generalization of classical linear models is varying coefficient models, which offer a flexible approach to modeling nonlinearity between covariates. A method of local weighted composite quantile regression is suggested to estimate the coefficient functions. The local Bahadur representation of the local estimator is derived and the asymptotic normality of the resulting estimator is established. Comparing to the local least squares estimator, the asymptotic relative efficiency is examined for the local weighted composite quantile estimator. Both theoretical analysis and numerical simulations reveal that the local weighted composite quantile estimator can obtain more efficient than the local least squares estimator for various non-normal errors. In the normal error case, the local weighted composite quantile estimator is almost as efficient as the local least squares estimator. Monte Carlo results are consistent with our theoretical findings. An empirical application demonstrates the potential of the proposed method.