Asymptotic distribution of the weighted least squares estimator

This paper derives the asymptotic distribution of the weighted least squares estimator (WLSE) in a heteroscedastic linear regression model. A consistent estimator of the asymptotic covariance matrix of the WLSE is also obtained. The results are obtained under weak conditions on the design matrix and some moment conditions on the error distributions. It is shown that most of the error distributions encountered in practice satisfy these moment conditions. Some examples of the asymptotic covariance matrices are also given.     

泛最小二乘法的改进及其容许性

考虑线性回归模型,当设计阵呈病态或秩亏时,我们用泛最小二乘法给出参数的估计,并证明其容许性;然后针对泛最小二乘估计对最小二乘估计过度压缩的缺点加以改进,使之更合理,有效.     

Error estimation for a weighted minimum-norm least squares solution with positive definite weights

The weighted least squares problem is considered. Given a generally inconsistent system of linear algebraic equations, error estimates are obtained for its weighted minimum-norm least squares solution under perturbations of the matrix and the right-hand side, including the case of rank modifications of the perturbed matrix.     

Asymptotic theory of least squares estimator in a nonregular nonlinear regression model

The asymptotic properties of the least squares estimator are derived for a nonregular nonlinear model via the study of weak convergence of the least squares process. This approach was adapted earlier by the author in the smooth case. The model discussed here is not amenable to analysis via the normal equations and Taylor expansions used by earlier authors.     

The distribution of a generalized least squares estimator with covariance adjustment

The distribution theory is developed for a generalized least squares estimator of the growth curve model. A special case of the estimator is the maximum likelihood estimator which is weighted by the sample covariance matrix. The distribution of two conditional forms of the estimator are derived and from these its density is obtained. Two general pivots and their distributions are derived from the conditional forms and special cases of these are investigated. The results obtained are linked to carlier work.     

约束最小二乘估计的优良性

对于线性模型y=Xθ ε,ε服从椭球等高单峰分布,未知参数θ满足不等式约束a′θ≥0,证明了在参数估计优良性的集中概率准则下,θ的约束最小二乘估计θ~*优于最小二乘估计θ.     

The rate of convergence for the least squares estimator in nonlinear regression model with dependent errors

We study the parameter estimation in a nonlinear regression model with a general error's structure,strong consistency and strong consistency rate of the least squares estimator are obtained.     

Asymptotic bias of the least squares estimator for multivariate autoregressive models

Summary The asymptotic bias of the least squares estimator for the multivariate autoregressive models is derived. The formulas for the low order univariate autoregressive models are given in terms of the simple functions of parameters. Our results are useful to the bias correction method of the least squares estimation. This work was supported by National Science Foundation Grant SES79-13976 at the Institute for Mathematical Studies in the Social Sciences, Stanford University. This paper is a revision of Discussion Paper No. 504, The Center for Mathematical Studies in Economics and Management Science, Northwestern University, October 1981.     

Minimax estimation and the least squares method

This paper is devoted to the problem of minimax estimation of parameters in linear regression models with uncertain second order statistics. The solution to the problem is shown to be the least squares estimator corresponding to the least favourable matrix of the second moments. This allows us to construct a new algorithm for minimax estimation closely connected with the least squares method. As an example, we consider the problem of polynomial regression introduced by A. N. Kolmogorov     

Perturbation analysis for mixed least squares–total least squares problems

In many linear parameter estimation problems, one can use the mixed least squares–total least squares (MTLS) approach to solve them. This paper is devoted to the perturbation analysis of the MTLS problem. Firstly, we present the normwise, mixed, and componentwise condition numbers of the MTLS problem, and find that the normwise, mixed, and componentwise condition numbers of the TLS problem and the LS problem are unified in the ones of the MTLS problem. In the analysis of the first‐order perturbation, we first provide an upper bound based on the normwise condition number. In order to overcome the problems encountered in calculating the normwise condition number, we give an upper bound for computing more effectively for the MTLS problem. As two estimation techniques for solving the linear parameter estimation problems, interesting connections between their solutions, their residuals for the MTLS problem, and the LS problem are compared. Finally, some numerical experiments are performed to illustrate our results.     

Latent scale linear models for multivariate ordinal responses and analysis by the method of weighted least squares

Summary A multivariate latent scale linear model is defined for multivariate ordered categorical responses and inference procedures based on the weighted least squares method are developed. Several applications of the model are suggested and illustrated through an analysis of real data. Asymptotic properties of the weighted least squares method are examined and some consequences of misspecification of the model are also discussed.     

On nonlinear weighted total least squares parameter estimation problem for the three-parameter Weibull density

The three-parameter Weibull density function is widely employed as a model in reliability and lifetime studies. Estimation of its parameters has been approached in the literature by various techniques, because a standard maximum likelihood estimate does not exist. In this paper we consider the nonlinear weighted total least squares fitting approach. As a main result, a theorem on the existence of the total least squares estimate is obtained, as well as its generalization in the total _lq$l_q$ norm (q?1$q?1$). Some numerical simulations to support the theoretical work are given.     

Parallel variable distribution for total least squares

A novel parallel method for determining an approximate total least squares (TLS) solution is introduced. Based on domain decomposition, the global TLS problem is partitioned into several dependent TLS subproblems. A convergent algorithm using the parallel variable distribution technique (SIAM J. Optim. 1994; 4 :815–832) is presented. Numerical results support the development and analysis of the algorithms. Copyright © 2005 John Wiley & Sons, Ltd.     

A weighted least squares method for scattered data fitting

In this paper, we present a weighted least squares method to fit scattered data with noise. Existence and uniqueness of a solution are proved and an error bound is derived. The numerical experiments illustrate that our weighted least squares method has better performance than the traditional least squares method in case of noisy data.     

Consistency conditions on the least squares estimator in single common factor analysis model

Summary This paper is concerned with the consistency of estimators in a single common factor analysis model when the dimension of the observed vector is not fixed. In the model several conditions on the sample sizen and the dimensionp are established for the least squares estimator (L.S.E.) to be consistent. Under some assumptions,p/n→0 is a necessary and sufficient condition that the L.S.E. converges in probability to the true value. A sufficient condition for almost sure convergence is also given.     

The condition numbers for weighted Moore-Penrose inverse and weighted linear least squares problem

Condition numbers play an important role in numerical analysis. Classical normise condition numbers are used to measure the size of both input perturbations and output errors. In this paper, we study the weighted normwise relative condition numbers for the weighted Moore-Penrose inverse and the weighted linear least-squares (WLS) problems in the case of the full-column rank matrix. The bounds or formulas for the weighted condition numbers are presented. The obtained results can be viewed as extensions of the earlier works studied by others.     

Abnormal behavior of the least squares estimate of multiple regression

An example is given to reveal the abnormal behavior of the least squares estimate of multiple regression. It is shown that the least squares estimate of the multiple linear regression may be “improved” in the sense of weak consistency when nuisance parameters are introduced into the model. A discussion on the implications of this finding is given.     

Numerical approximation of the Oldroyd‐B model by the weighted least‐squares/discontinuous Galerkin method

We consider a numerical method for the Oldroyd‐B model of viscoelastic fluid flows by a combination of the weighted least‐squares (WLS) method and the discontinuous Galerkin (DG) finite element method. The constitutive equation is decoupled from the momentum and continuity equations, and the approximate solution is computed iteratively by solving the Stokes problem and a linearized constitutive equation using WLS and DG, respectively. An a priori error estimate for the WLS/DG method is derived and numerical results supporting the estimate are presented. © 2012 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq 2013     

Iterative weighted partial spline least squares estimation in semiparametric modeling of longitudinal data

In this paper we consider the estimating problem of a semiparametric regression modelling whenthe data are longitudinal.An iterative weighted partial spline least squares estimator(IWPSLSE)for the para-metric component is proposed which is more efficient than the weighted partial spline least squares estimator(WPSLSE)with weights constructed by using the within-group partial spline least squares residuals in the sense