Linear least squares problems with data over incomplete grids

The paper addresses bivariate surface fitting problems, where data points lie on the vertices of a rectangular grid. Efficient and stable algorithms can be found in the literature to solve such problems. If data values are missing at some grid points, there exists a computational method for finding a least squares spline by fixing appropriate values for the missing data. We extended this technique to arbitrary least squares problems as well as to linear least squares problems with linear equality constraints. Numerical examples are given to show the effectiveness of the technique presented. AMS subject classification (2000)  65D05, 65D07, 65D10, 65F05, 65F20     

Circle fitting by linear and nonlinear least squares

The problem of determining the circle of best fit to a set of points in the plane (or the obvious generalization ton-dimensions) is easily formulated as a nonlinear total least-squares problem which may be solved using a Gauss-Newton minimization algorithm. This straight-forward approach is shown to be inefficient and extremely sensitive to the presence of outliers. An alternative formulation allows the problem to be reduced to a linear least squares problem which is trivially solved. The recommended approach is shown to have the added advantage of being much less sensitive to outliers than the nonlinear least squares approach.This work was completed while the author was visiting the Numerical Optimisation Centre, Hatfield Polytechnic and benefitted from the encouragement and helpful suggestions of Dr. M. C. Bartholomew-Biggs and Professor L. C. W. Dixon.     

Least Squares Solution of Bivariate Surface Fitting Problems Using Tensor Product Splines

The paper treats bivariate surface fitting problems, where the data points lie on lines parallel to one of the axes. The associated bivariate collocation matrix is investigated as a block Kronecker product of univariate collocation matrices. Based on various properties of this block Kronecker product, such scattered data are characterized where the associated interpolation problem using tensor product splines admits a unique solution.     

基于最小二乘近似的模型平均方法

本文提出基于最小二乘近似的模型平均方法.该方法可用于线性模型、广义线性模型和分位数回归等各种常用模型.特别地,经典的Mallows模型平均方法是该方法的特例.现存的模型平均文献中,渐近分布的证明一般需要局部误设定假设,所得的极限分布的形式也比较复杂.本文将在不使用局部误设定假设的情形下证明该方法的渐近正态性.另外,本文...     

On stable least squares solution to the system of linear inequalities

The system of inequalities is transformed to the least squares problem on the positive ortant. This problem is solved using orthogonal transformations which are memorized as products. Author’s previous paper presented a method where at each step all the coefficients of the system were transformed. This paper describes a method applicable also to large matrices. Like in revised simplex method, in this method an auxiliary matrix is used for the computations. The algorithm is suitable for unstable and degenerate problems primarily.      

Solution of linear least squares via the ABS algorithm

The ABS class for linear and nonlinear systems has been recently introduced by Abaffy, Broyden, Galantai and Spedicato. Here we consider various ways of applying these algorithms to the determination of the minimal euclidean norm solution of over-determined linear systems in the least squares sense. Extensive numerical experiments show that the proposed algorithms are efficient and that one of them usually gives better accuracy than standard implementations of the QR orthogonalization algorithm with Householder reflections.     

An algorithm for least squares analysis of spectroscopic data

This paper describes a variant of the Gauss-Newton-Hartley algorithm for nonlinear least squares, in which aQR implementation is used to solve the linear least squares problem. We follow Grey's idea of updating variables at intermediate stages of the orthogonalization. This technique, applied in partitions identified with known or suspected spectral lines, appears to be especially suited to the analysis of spectroscopic data. We suggest that this algorithm is an attractive candidate for the optimization role in Ekenberg's interactive computer graphics curve fitting program.     

The condition numbers for weighted Moore-Penrose inverse and weighted linear least squares problem

Condition numbers play an important role in numerical analysis. Classical normise condition numbers are used to measure the size of both input perturbations and output errors. In this paper, we study the weighted normwise relative condition numbers for the weighted Moore-Penrose inverse and the weighted linear least-squares (WLS) problems in the case of the full-column rank matrix. The bounds or formulas for the weighted condition numbers are presented. The obtained results can be viewed as extensions of the earlier works studied by others.     

Implicitly-weighted total least squares

In a total least squares (TLS) problem, we estimate an optimal set of model parameters X, so that (A-ΔA)X=B-ΔB, where A is the model matrix, B is the observed data, and ΔA and ΔB are corresponding corrections. When B is a single vector, Rao (1997) and Paige and Strakoš (2002) suggested formulating standard least squares problems, for which ΔA=0, and data least squares problems, for which ΔB=0, as weighted and scaled TLS problems. In this work we define an implicitly-weighted TLS formulation (ITLS) that reparameterizes these formulations to make computation easier. We derive asymptotic properties of the estimates as the number of rows in the problem approaches infinity, handling the rank-deficient case as well. We discuss the role of the ratio between the variances of errors in A and B in choosing an appropriate parameter in ITLS. We also propose methods for computing the family of solutions efficiently and for choosing the appropriate solution if the ratio of variances is unknown. We provide experimental results on the usefulness of the ITLS family of solutions.     

Bivariate Free Knot Splines

We consider the least squares approximation of gridded 2D data by tensor product splines with free knots. The smoothing functional to be minimized—a generalization of the univariate Schoenberg functional—is chosen in such a way that the solution of the bivariate problem separates into the solution of a sequence of univariate problems in case of fixed knots. The resulting optimization problem is a constrained separable least squares problem with tensor product structure. Based on some ideas developed by the authors for the univariate case, an efficient method for solving the specially structured 2D problem is proposed, analyzed and tested on hand of some examples from the literature.     

Least squares approximation by splines with free knots

Suppose we are given noisy data which are considered to be perturbed values of a smooth, univariate function. In order to approximate these data in the least squares sense, a linear combination of B-splines is used where the tradeoff between smoothness and closeness of the fit is controlled by a smoothing term which regularizes the least squares problem and guarantees unique solvability independent of the position of knots. Moreover, a subset of the knot sequence which defines the B-splines, the so-calledfree knots, is included in the optimization process.The resulting constrained least squares problem which is linear in the spline coefficients but nonlinear in the free knots is reduced to a problem that has only the free knots as variables. The reduced problem is solved by a generalized Gauss-Newton method. The method developed can be combined with a knot removal strategy in order to obtain an approximating spline with as few parameters as possible.Dedicated to Professor Dr.-Ing. habil. Dr. h.c. Helmut Heinrich on the occasion of his 90th birthdayResearch of the second author was partly supported by Deutsche Forschungsgemeinschaft under grant Schm 968/2-1.     

On nonlinear weighted total least squares parameter estimation problem for the three-parameter Weibull density

The three-parameter Weibull density function is widely employed as a model in reliability and lifetime studies. Estimation of its parameters has been approached in the literature by various techniques, because a standard maximum likelihood estimate does not exist. In this paper we consider the nonlinear weighted total least squares fitting approach. As a main result, a theorem on the existence of the total least squares estimate is obtained, as well as its generalization in the total _lq$l_q$ norm (q?1$q?1$). Some numerical simulations to support the theoretical work are given.     

Linear restrictions and two step least squares with applications

In this paper we consider the full rank regression model with arbitrary covariance matrix: Y = Xß + ε. It is shown that the effect of restricting the information Y to T = A′Y may be analyzed through an associatedi regression problem which is amenable to solution by two step least squares. The results are applied to the important case of missing observations, where some classical results are rederived.     

局部平稳扩散模型中时变参数的加权最小二乘估计

基于离散观测样本,利用局部线性拟合,得到了局部平稳扩散模型中时变漂移参数的加权最小二乘估计,并讨论了估计量的相合性,渐近正态性和一致收敛速度.同时,通过模拟研究说明了估计量的有效性.     

On mixed and componentwise condition numbers for Moore-Penrose inverse and linear least squares problems

Classical condition numbers are normwise: they measure the size of both input perturbations and output errors using some norms. To take into account the relative of each data component, and, in particular, a possible data sparseness, componentwise condition numbers have been increasingly considered. These are mostly of two kinds: mixed and componentwise. In this paper, we give explicit expressions, computable from the data, for the mixed and componentwise condition numbers for the computation of the Moore-Penrose inverse as well as for the computation of solutions and residues of linear least squares problems. In both cases the data matrices have full column (row) rank.

     

Local results for the Gauss-Newton method on constrained rank-deficient nonlinear least squares

A nonlinear least squares problem with nonlinear constraints may be ill posed or even rank-deficient in two ways. Considering the problem formulated as subject to the constraints , the Jacobian and/or the Jacobian , , may be ill conditioned at the solution.

We analyze the important special case when and/or do not have full rank at the solution. In order to solve such a problem, we formulate a nonlinear least norm problem. Next we describe a truncated Gauss-Newton method. We show that the local convergence rate is determined by the maximum of three independent Rayleigh quotients related to three different spaces in .

Another way of solving an ill-posed nonlinear least squares problem is to regularize the problem with some parameter that is reduced as the iterates converge to the minimum. Our approach is a Tikhonov based local linear regularization that converges to a minimum norm problem. This approach may be used both for almost and rank-deficient Jacobians.

Finally we present computational tests on constructed problems verifying the local analysis.

     

On polynomial approximation in the uniform norm by the discrete least squares method

The discrete least squares method is convenient for computing polynomial approximations to functions. We investigate the possibility of using this method to obtain polynomial approximants good in the uniform norm, and find that for a given set ofm nodes, the degreen of the approximating polynomial should be selected so that there is a subset ofn+1 nodes which are close ton+1 Fejér points for the curve. Numerical examples are presented.Sponsored by the United States Army under Contract No. DAAG29-80-C-0041.     

Latent scale linear models for multivariate ordinal responses and analysis by the method of weighted least squares

Summary A multivariate latent scale linear model is defined for multivariate ordered categorical responses and inference procedures based on the weighted least squares method are developed. Several applications of the model are suggested and illustrated through an analysis of real data. Asymptotic properties of the weighted least squares method are examined and some consequences of misspecification of the model are also discussed.     

On the perturbation of least squares problems with equality constraints

Some new perturbation results are presented for least squares problems with equality constraints, in which relative errors are obtained on perturbed solutions, least squares residuals, and vectors of Lagrange multipliers of the problem, based on the equivalence of the problem to a usual least squares problem and optimal perturbation results for orthogonal projections.     

Multigrid for the Galerkin least squares method in linear elasticity

In SIAM J. Numer. Anal. 28 (1991) 1680-1697, Franca and Stenberg developed several Galerkin least squares methods for the solution of the problem of linear elasticity. That work concerned itself only with the error estimates of the method. It did not address the related problem of finding effective methods for the solution of the associated linear systems. In this work, we prove the convergence of a multigrid method. This multigrid is robust in that the convergence is uniform as the parameter ν goes to 1/2. Computational experiments are included.