Detection of outliers in weighted least squares regression

In multiple linear regression model, we have presupposed assumptions (independence, normality, variance homogeneity and so on) on error term. When case weights are given because of variance heterogeneity, we can estimate efficiently regression parameter using weighted least squares estimator. Unfortunately, this estimator is sensitive to outliers like ordinary least squares estimator. Thus, in this paper, we proposed some statistics for detection of outliers in weighted least squares regression.     

基于泛岭估计对岭估计过度压缩的改进方法

岭估计是解决多元线性回归多重共线性问题的有效方法,是有偏的压缩估计。与普通最小二乘估计相比,岭估计可以降低参数估计的均方误差,但是却增大残差平方和,拟合效果变差。本文提出一种基于泛岭估计对岭估计过度压缩的改进方法,可以改进岭估计的拟合效果,减小岭估计残差平方和的增加幅度。     

半参数回归模型的稳健补偿最小二乘估计

本文研究了一类半参数回归模型，利用稳健补偿最小二乘估计法，得到了稳健补偿最小二乘估计量，以及它们的影响函数及渐近方差一协方差，对结果的分析表明了该法优于补偿最小二乘法，而且具有稳定性．     

PLS classification of functional data

Partial least squares (PLS) approach is proposed for linear discriminant analysis (LDA) when predictors are data of functional type (curves). Based on the equivalence between LDA and the multiple linear regression (binary response) and LDA and the canonical correlation analysis (more than two groups), the PLS regression on functional data is used to estimate the discriminant coefficient functions. A simulation study as well as an application to kneading data compare the PLS model results with those given by other methods.     

Estimation and construction of confidence regions in regression models

In the present paper, linear and nonlinear regression models with a growing number of unknown parameters are considered. Conditions sufficient for the least squares estimators to be consistent are formulated. Estimates for the covariance matrix of least squares estimators, which make it possible to construct confidence regions for the regression function, are given. Translated fromStatisticheskie Metody Otsenivaniya i Proverki Gipotez, pp. 142–150, Perm, 1990.     

Second-order nonlinear least squares estimation

The ordinary least squares estimation is based on minimization of the squared distance of the response variable to its conditional mean given the predictor variable. We extend this method by including in the criterion function the distance of the squared response variable to its second conditional moment. It is shown that this “second-order” least squares estimator is asymptotically more efficient than the ordinary least squares estimator if the third moment of the random error is nonzero, and both estimators have the same asymptotic covariance matrix if the error distribution is symmetric. Simulation studies show that the variance reduction of the new estimator can be as high as 50% for sample sizes lower than 100. As a by-product, the joint asymptotic covariance matrix of the ordinary least squares estimators for the regression parameter and for the random error variance is also derived, which is only available in the literature for very special cases, e.g. that random error has a normal distribution. The results apply to both linear and nonlinear regression models, where the random error distributions are not necessarily known.     

Some observations on local least squares

In previous work we introduced a construction to produce biorthogonal multiresolutions from given subdivisions. The approach involved estimating the solution to a least squares problem by means of a number of smaller least squares approximations on local portions of the data. In this work we use a result by Dahlquist, et al. on the method of averages to make observational comparisons between this local least squares estimation and full least squares approximation. We have explored examples in two problem domains: data reduction and data approximation. We observe that, particularly for design matrices with a repetitive pattern of column entries, the least squares solution is often well estimated by local least squares, that the estimation rapidly improves with the size of the local least squares problems, and that the quality of the estimate is largely independent of the size of the full problem. In memory of Germund Dahlquist (1925–2005).AMS subject classification (2000) 93E24     

应用SAS解非线性回归问题

.应用SAS/STAT估计非线性回归模型中的参数.首先,通过变量代换,把可以线性化的非线性回归模型化为线性回归模型,并用普通最小二乘法、主成分分析法和偏最小二乘法求模型中的参数和回归模型.其次,通过改良的高斯—牛顿迭代法来估计Logistic模型和Compertz模型中的参数.     

Spline Estimation of Discontinuous Regression Functions

Abstract

This article deals with regression function estimation when the regression function is smooth at all but a finite number of points. An important question is: How can one produce discontinuous output without knowledge of the location of discontinuity points? Unlike most commonly used smoothers that tend to blur discontinuity in the data, we need to find a smoother that can detect such discontinuity. In this article, linear splines are used to estimate discontinuous regression functions. A procedure of knot-merging is introduced for the estimation of regression functions near discontinuous points. The basic idea is to use multiple knots for spline estimates. We use an automatic procedure involving the least squares method, stepwise knot addition, stepwise basis deletion, knot-merging, and the Bayes information criterion to select the final model. The proposed method can produce discontinuous outputs. Numerical examples using both simulated and real data are given to illustrate the performance of the proposed method.     

Convergence systems and strong consistency of least squares estimates in regression models

A recent theorem of T. L. Hai, H. Robbins, and C. Z. Wei (J. Multivariate Anal.9 (1979), 343–362) is extended to a more general form which unifies previous results in the literature on the strong consistency of least squares estimates in multiple regression models with nonrandom regressors. In particular the issue of strong consistency of the least squares estimate in the Gauss-Markov model, in the i.i.d. model with infinite second moment, and in general time series models is examined. In this connection, some basic properties of convergence systems are also obtained and are applied to the strong consistency problem.     

齐次线性约束回归模型参数的主成分压缩估计

当设计矩阵X复共线时,对齐次线性约束回归模型参数的约束最小二乘估计进行改进,提出参数的主成分压缩估计,并对新参数估计的性质进行了讨论,最后进行了数值模拟,验证了算法的参数估计优于约束最小二乘估计.     

Iterative Weighted Semiparametric Least Squares Estimation in Repeated Measurement Partially Linear Regression Models

Consider a repeated measurement partially linear regression model with an unknown vector parameter β, an unknown function g(.), and unknown heteroscedastic error variances. In order to improve the semiparametric generalized least squares estimator (SGLSE) of β, we propose an iterative weighted semiparametric least squares estimator (IWSLSE) and show that it improves upon the SGLSE in terms of asymptotic covariance matrix. An adaptive procedure is given to determine the number of iterations. We also show that when the number of replicates is less than or equal to two, the IWSLSE can not improve upon the SGLSE. These results are generalizations of those in [2] to the case of semiparametric regressions.     

Multiple criteria linear regression

The unknown parameters in multiple linear regression models may be estimated using any one of a number of criteria such as the minimization of the sum of squared errors MSSE, the minimization of the sum of absolute errors MSAE, and the minimization of the maximum absolute error MMAE. At present, the MSSE or the least squares criterion continues to be the most popular. However, at times the choice of a criterion is not clear from statistical, practical or other considerations. Under such circumstances, it may be more appropriate to use multiple criteria rather than a single criterion to estimate the unknown parameters in a multiple linear regression model. We motivate the use of multiple criteria estimation in linear regression models with an example, propose a few models, and outline a solution procedure.     

Least squares estimation of linear regression models for convex compact random sets

Simple and multiple linear regression models are considered between variables whose “values” are convex compact random sets in ${\mathbb{R}^p}$ , (that is, hypercubes, spheres, and so on). We analyze such models within a set-arithmetic approach. Contrary to what happens for random variables, the least squares optimal solutions for the basic affine transformation model do not produce suitable estimates for the linear regression model. First, we derive least squares estimators for the simple linear regression model and examine them from a theoretical perspective. Moreover, the multiple linear regression model is dealt with and a stepwise algorithm is developed in order to find the estimates in this case. The particular problem of the linear regression with interval-valued data is also considered and illustrated by means of a real-life example.     

天然气消费量的偏最小二乘支持向量机预测

结合偏最小二乘法和支持向量机的优缺点,提出基于偏最小二乘支持向量机的天然气消费量预测模型。首先,利用偏最小二乘法确定影响天然气消费量的新综合变量,建立以新综合变量为输入,天然气消费量为输出的支持向量机模型,对天然气消费量进行了预测;然后,与多元回归、偏最小二乘回归、普通支持向量机做误差检验比较,验证该方法的可行性与正确性。结果表明,此天然气消费量预测模型具有较高的精确度和应用价值。     

变系数模型变窗宽局部M-估计的相合性

用变窗宽局部M-估计对变系数模型的系数函数进行估计,得到了估计的相合性和渐近正态性.所采用的方法继承了局部多项式回归的优点并且克服了最小二乘方法缺乏稳健性的缺点.变窗宽的使用提高了局部M-估计的可塑性,并使得它们能成功地处理空间非齐性曲线、异方差性及非均匀设计密度.     

Universal Algorithms for Learning Theory. Part II: Piecewise Polynomial Functions

This paper is concerned with estimating the regression function f_ρ in supervised learning by utilizing piecewise polynomial approximations on adaptively generated partitions. The main point of interest is algorithms that with high probability are optimal in terms of the least square error achieved for a given number m of observed data. In a previous paper [1], we have developed for each β > 0 an algorithm for piecewise constant approximation which is proven to provide such optimal order estimates with probability larger than 1- m^-β. In this paper we consider the case of higher-degree polynomials. We show that for general probability measures ρ empirical least squares minimization will not provide optimal error estimates with high probability. We go further in identifying certain conditions on the probability measure ρ which will allow optimal estimates with high probability.     

A semiparametric method for estimating nonlinear autoregressive model with dependent errors

The first-order nonlinear autoregressive model is considered and a semiparametric method is proposed to estimate regression function. In the presented model, dependent errors are defined as first-order autoregressive AR(1). The conditional least squares method is used for parametric estimation and the nonparametric kernel approach is applied to estimate regression adjustment. In this case, some asymptotic behaviors and simulated results for the semiparametric method are presented. Furthermore, the method is applied for the financial data in Iran’s Tejarat-Bank.     

An estimate of the convergence rate in the central limit theorem for two-parameter martingale differences

We present an estimate of the rate of convergence to the normal law of the least squares estimator of the regression coefficient for a random field which is a two-parameter martingale difference sequence.Translated from Ukrainskii Matematicheskii Zhurnal, Vol. 44, No. 8, pp. 1138–1141, August, 1992.     

Backward perturbation analysis for scaled total least‐squares problems

The scaled total least‐squares (STLS) method unifies the ordinary least‐squares (OLS), the total least‐squares (TLS), and the data least‐squares (DLS) methods. In this paper we perform a backward perturbation analysis of the STLS problem. This also unifies the backward perturbation analyses of the OLS, TLS and DLS problems. We derive an expression for an extended minimal backward error of the STLS problem. This is an asymptotically tight lower bound on the true minimal backward error. If the given approximate solution is close enough to the true STLS solution (as is the goal in practice), then the extended minimal backward error is in fact the minimal backward error. Since the extended minimal backward error is expensive to compute directly, we present a lower bound on it as well as an asymptotic estimate for it, both of which can be computed or estimated more efficiently. Our numerical examples suggest that the lower bound gives good order of magnitude approximations, while the asymptotic estimate is an excellent estimate. We show how to use our results to easily obtain the corresponding results for the OLS and DLS problems in the literature. Copyright © 2009 John Wiley & Sons, Ltd.