A weighted pseudoinverse,generalized singular values,and constrained least squares problems

The weighted pseudoinverse providing the minimum semi-norm solution of the weighted linear least squares problem is studied. It is shown that it has properties analogous to those of the Moore-Penrose pseudoinverse. The relation between the weighted pseudoinverse and generalized singular values is explained. The weighted pseudoinverse theory is used to analyse least squares problems with linear and quadratic constraints. A numerical algorithm for the computation of the weighted pseudoinverse is briefly described.This work was supported in part by the Swedish Institute for Applied Mathematics.     

Asymptotic distribution of the weighted least squares estimator

This paper derives the asymptotic distribution of the weighted least squares estimator (WLSE) in a heteroscedastic linear regression model. A consistent estimator of the asymptotic covariance matrix of the WLSE is also obtained. The results are obtained under weak conditions on the design matrix and some moment conditions on the error distributions. It is shown that most of the error distributions encountered in practice satisfy these moment conditions. Some examples of the asymptotic covariance matrices are also given.     

On the condition number of linear least squares problems in a weighted Frobenius norm

LetA be anm × n, m n full rank real matrix andb a real vector of sizem. We give in this paper an explicit formula for the condition number of the linear least squares problem (LLSP) defined by min Ax–b₂,x ⁿ . Let and be two positive real numbers, we choose the weighted Frobenius norm [A, b] _F on the data and the usual Euclidean norm on the solution. A straightforward generalization of the backward error of [9] to this norm is also provided. This allows us to carry out a first order estimate of the forward error for the LLSP with this norm. This enables us to perform a complete backward error analysis in the chosen norms.Finally, some numerical results are presented in the last section on matrices from the collection of [5]. Three algorithms have been tested: the QR factorization, the Normal Equations (NE), the Semi-Normal Equations (SNE).     

The condition numbers for weighted Moore-Penrose inverse and weighted linear least squares problem

Condition numbers play an important role in numerical analysis. Classical normise condition numbers are used to measure the size of both input perturbations and output errors. In this paper, we study the weighted normwise relative condition numbers for the weighted Moore-Penrose inverse and the weighted linear least-squares (WLS) problems in the case of the full-column rank matrix. The bounds or formulas for the weighted condition numbers are presented. The obtained results can be viewed as extensions of the earlier works studied by others.     

On nonlinear weighted total least squares parameter estimation problem for the three-parameter Weibull density

The three-parameter Weibull density function is widely employed as a model in reliability and lifetime studies. Estimation of its parameters has been approached in the literature by various techniques, because a standard maximum likelihood estimate does not exist. In this paper we consider the nonlinear weighted total least squares fitting approach. As a main result, a theorem on the existence of the total least squares estimate is obtained, as well as its generalization in the total _lq$l_q$ norm (q?1$q?1$). Some numerical simulations to support the theoretical work are given.     

Least‐squares solutions of matrix inverse problem for bi‐symmetric matrices with a submatrix constraint

An n × n real matrix A = (a_ij)_{n × n} is called bi‐symmetric matrix if A is both symmetric and per‐symmetric, that is, a_ij = a_ji and a_ij = a_n+1?1,n+1?i (i, j = 1, 2,..., n). This paper is mainly concerned with finding the least‐squares bi‐symmetric solutions of matrix inverse problem AX = B with a submatrix constraint, where X and B are given matrices of suitable sizes. Moreover, in the corresponding solution set, the analytical expression of the optimal approximation solution to a given matrix A^* is derived. A direct method for finding the optimal approximation solution is described in detail, and three numerical examples are provided to show the validity of our algorithm. Copyright © 2007 John Wiley & Sons, Ltd.     

Perturbation analysis for mixed least squares–total least squares problems

In many linear parameter estimation problems, one can use the mixed least squares–total least squares (MTLS) approach to solve them. This paper is devoted to the perturbation analysis of the MTLS problem. Firstly, we present the normwise, mixed, and componentwise condition numbers of the MTLS problem, and find that the normwise, mixed, and componentwise condition numbers of the TLS problem and the LS problem are unified in the ones of the MTLS problem. In the analysis of the first‐order perturbation, we first provide an upper bound based on the normwise condition number. In order to overcome the problems encountered in calculating the normwise condition number, we give an upper bound for computing more effectively for the MTLS problem. As two estimation techniques for solving the linear parameter estimation problems, interesting connections between their solutions, their residuals for the MTLS problem, and the LS problem are compared. Finally, some numerical experiments are performed to illustrate our results.     

A preconditioner for constrained and weighted least squares problems with Toeplitz structure

We study methods for solving the constrained and weighted least squares problem min _x by the preconditioned conjugate gradient (PCG) method. HereW = diag (₁, , _m ) with _{1 m} 0, andA ^T = [T ₁ ^T , ,T _k ^T ] with Toeplitz blocksT _l R ^{n × n} ,l = 1, ,k. It is well-known that this problem can be solved by solving anaugmented linear 2 × 2 block linear systemM +Ax =b, A ^T = 0, whereM =W ^–1. We will use the PCG method with circulant-like preconditioner for solving the system. We show that the spectrum of the preconditioned matrix is clustered around one. When the PCG method is applied to solve the system, we can expect a fast convergence rate.Research supported by HKRGC grants no. CUHK 178/93E and CUHK 316/94E.     

Perturbation theory for generalized and constrained linear least squares

The perturbation analysis of weighted and constrained rank‐deficient linear least squares is difficult without the use of the augmented system of equations. In this paper a general form of the augmented system is used to get simple perturbation identities and perturbation bounds for the general linear least squares problem both for the full‐rank and rank‐deficient problem. Perturbation identities for the rank‐deficient weighted and constrained case are found as a special case. Interesting perturbation bounds and condition numbers are derived that may be useful when considering the stability of a solution of the rank‐deficient general least squares problem. Copyright © 2000 John Wiley & Sons, Ltd.     

A condition analysis of the weighted linear least squares problem using dual norms

In this paper, based on the theory of adjoint operators and dual norms, we define condition numbers for a linear solution function of the weighted linear least squares problem. The explicit expressions of the normwise and componentwise condition numbers derived in this paper can be computed at low cost when the dimension of the linear function is low due to dual operator theory. Moreover, we use the augmented system to perform a componentwise perturbation analysis of the solution and residual of the weighted linear least squares problems. We also propose two efficient condition number estimators. Our numerical experiments demonstrate that our condition numbers give accurate perturbation bounds and can reveal the conditioning of individual components of the solution. Our condition number estimators are accurate as well as efficient.     

Iterative weighted partial spline least squares estimation in semiparametric modeling of longitudinal data

In this paper we consider the estimating problem of a semiparametric regression modelling whenthe data are longitudinal.An iterative weighted partial spline least squares estimator(IWPSLSE)for the para-metric component is proposed which is more efficient than the weighted partial spline least squares estimator(WPSLSE)with weights constructed by using the within-group partial spline least squares residuals in the sense     

Randomized algorithms for total least squares problems

Motivated by the recently popular probabilistic methods for low‐rank approximations and randomized algorithms for the least squares problems, we develop randomized algorithms for the total least squares problem with a single right‐hand side. We present the Nyström method for the medium‐sized problems. For the large‐scale and ill‐conditioned cases, we introduce the randomized truncated total least squares with the known or estimated rank as the regularization parameter. We analyze the accuracy of the algorithm randomized truncated total least squares and perform numerical experiments to demonstrate the efficiency of our randomized algorithms. The randomized algorithms can greatly reduce the computational time and still maintain good accuracy with very high probability.     

Approximating minimum norm solutions of rank-deficient least squares problems

In this paper we consider the solution of linear least squares problems min_x∥Ax - b∥²₂ where the matrix A ∈ R ^{m × n} is rank deficient. Put p = min{m, n}, let σ_i, i = 1, 2,…, p, denote the singular values of A, and let u_i and v_i denote the corresponding left and right singular vectors. Then the minimum norm solution of the least squares problem has the form x* = ∫^r_{i = 1}(u^T_ib/σ_i)v_i, where r ≤ p is the rank of A. The Riley–Golub iteration, x_{k + 1} = arg min_x{∥Ax - b∥²₂ + λ∥x − x_k∥²₂} converges to the minimum norm solution if x₀ is chosen equal to zero. The iteration is implemented so that it takes advantage of a bidiagonal decomposition of A. Thus modified, the iteration requires only O(p) flops (floating point operations). A further gain of using the bidiagonalization of A is that both the singular values σ_i and the scalar products u^T_ib can be computed at marginal extra cost. Moreover, we determine the regularization parameter, λ, and the number of iterations, k, in a way that minimizes the difference x* − x_k with respect to a certain norm. Explicit rules are derived for calculating these parameters. One advantage of our approach is that the numerical rank can be easily determined by using the singular values. Furthermore, by the iterative procedure, x* is approximated without computing the singular vectors of A. This gives a fast and reliable method for approximating minimum norm solutions of well-conditioned rank-deficient least squares problems. Numerical experiments illustrate the viability of our ideas, and demonstrate that the new method gives more accurate approximations than an approach based on a QR decomposition with column pivoting. © 1998 John Wiley & Sons, Ltd.     

Asymptotics for a weighted least squares estimator of the disease onset distribution function for a survival-sacrifice model

In carcinogenicity experiments with animals where the tumor is not palpable it is common to observe only the time of death of the animal, the cause of death (the tumor or another independent cause, as sacrifice) and whether the tumor was present at the time of death. These last two indicator variables are evaluated after an autopsy. A weighted least squares estimator for the distribution function of the disease onset was proposed. Asymptotic properties of that estimator are established here. We demonstrate its strong uniform consistency. A minimax lower bound for the estimation of the disease onset distribution is obtained, as well as the local asymptotic distribution for their estimator.     

Real structure-preserving algorithm for quaternion equality constrained least squares problem

Quaternion equality constrained least squares problem is an extremely effective tool in studying quantum mechanics and quantum field theory. However, the computation of the quaternion equality constrained least squares problem is extremely complex. In this paper, we first prove that quaternion equality constrained least squares problem is equivalent to weighted quaternion least squares problem when the parameter $\tau \to +\infty$. Then, for weighted quaternion least squares problem, applying the special structure of real representation of quaternion, we propose real structure–preserving algorithm to obtain the solution of quaternion equality contained least squares problem. At last, we give numerical examples to illustrate the effectiveness of our method.     

On the modified Gram-Schmidt algorithm for weighted and constrained linear least squares problems

A framework and an algorithm for using modified Gram-Schmidt for constrained and weighted linear least squares problems is presented. It is shown that a direct implementation of a weighted modified Gram-Schmidt algorithm is unstable for heavily weighted problems. It is shown that, in most cases it is possible to get a stable algorithm by a simple modification free from any extra computational costs. In particular, it is not necessary to perform reorthogonalization.Solving the weighted and constrained linear least squares problem with the presented weighted modified Gram-Schmidt algorithm is seen to be numerically equivalent to an algorithm based on a weighted Householder-likeQR factorization applied to a slightly larger problem. This equivalence is used to explain the instability of the weighted modified Gram-Schmidt algorithm. If orthogonality, with respect to a weighted inner product, of the columns inQ is important then reorthogonalization can be used. One way of performing such reorthogonalization is described.Computational tests are given to show the main features of the algorithm.