Using dual techniques to derive componentwise and mixed condition numbers for a linear function of a linear least squares solution

We prove duality results for adjoint operators and product norms in the framework of Euclidean spaces. We show how these results can be used to derive condition numbers especially when perturbations on data are measured componentwise relatively to the original data. We apply this technique to obtain formulas for componentwise and mixed condition numbers for a linear function of a linear least squares solution. These expressions are closed when perturbations of the solution are measured using a componentwise norm or the infinity norm and we get an upper bound for the Euclidean norm.      

On condition numbers for Moore–Penrose inverse and linear least squares problem involving Kronecker products

In this paper, we investigate the normwise, mixed, and componentwise condition numbers and their upper bounds for the Moore–Penrose inverse of the Kronecker product and more general matrix function compositions involving Kronecker products. We also present the condition numbers and their upper bounds for the associated Kronecker product linear least squares solution with full column rank. In practice, the derived upper bounds for the mixed and componentwise condition numbers for Kronecker product linear least squares solution can be efficiently estimated using the Hager–Higham Algorithm. Copyright © 2012 John Wiley & Sons, Ltd.     

COMPONENTWISE CONDITION NUMBERS FOR GENERALIZED MATRIX INVERSION AND LINEAR LEAST SQUARES

We present componentwise condition numbers for the problems of Moore-Penrose generalized matrix inversion and linear least squares. Also, the condition numbers for these condition numbers are given.     

A condition analysis of the weighted linear least squares problem using dual norms

In this paper, based on the theory of adjoint operators and dual norms, we define condition numbers for a linear solution function of the weighted linear least squares problem. The explicit expressions of the normwise and componentwise condition numbers derived in this paper can be computed at low cost when the dimension of the linear function is low due to dual operator theory. Moreover, we use the augmented system to perform a componentwise perturbation analysis of the solution and residual of the weighted linear least squares problems. We also propose two efficient condition number estimators. Our numerical experiments demonstrate that our condition numbers give accurate perturbation bounds and can reveal the conditioning of individual components of the solution. Our condition number estimators are accurate as well as efficient.     

带多重右边的不定最小二乘问题的条件数

本文研究了带多重右边的不定最小二乘问题的条件数,给出了范数型、混合型及分量型条件数的表达式,同时,也给出了相应的结构条件数的表达式.所考虑的结构矩阵包含Toeplitz 矩阵、Hankel矩阵、对称矩阵、三对角矩阵等线性结构矩阵与Vandermonde矩阵、Cauchy矩阵等非线性结构矩阵.数值例子显示结构条件数总是紧于非结构条件数.     

Stability and Sensitivity of Tridiagonal LU Factorization without Pivoting

In this paper the accuracy of LU factorization of tridiagonal matrices without pivoting is considered. Two types of componentwise condition numbers for the L and U factors of tridiadonal matrices are presented and compared. One type is a condition number with respect to small relative perturbations of each entry of the matrix. The other type is a condition number with respect to small componentwise perturbations of the kind appearing in the backward error analysis of the usual algorithm for the LU factorization. We show that both condition numbers are of similar magnitude. This means that the algorithm is componentwise forward stable, i.e., the forward errors are of similar magnitude to those produced by a componentwise backward stable method. Moreover the presented condition numbers can be computed in O(n) flops, which allows to estimate with low cost the forward errors. AMS subject classification (2000) 65F35, 65F50, 15A12, 15A23, 65G50.Received October 2003. Accepted August 2004. Communicated by Per Christian Hansen.Froilán M. Dopico: This research has been partially supported by the Ministerio de Ciencia y Tecnología of Spain through grants BFM2003-06335-C03-02 (M. I. Bueno) and BFM2000-0008 (F. M. Dopico).     

Ill-Conditionedness Need not be Componentwise Near to Ill-Posedness for Least Squares Problems

The condition number of a problem measures the sensitivity of the answer to small changes in the input, where small refers to some distance measure. A problem is called ill-conditioned if the condition number is large, and it is called ill-posed if the condition number is infinity. It is known that for many problems the (normwise) distance to the nearest ill-posed problem is proportional to the reciprocal of the condition number. Recently it has been shown that for linear systems and matrix inversion this is also true for componentwise distances. In this note we show that this is no longer true for least squares problems and other problems involving rectangular matrices. Problems are identified which are arbitrarily ill-conditioned (in a componentwise sense) whereas any componentwise relative perturbation less than 1 cannot produce an ill-posed problem. Bounds are given using additional information on the matrix.     

Minimum-Euclidean-norm matrix correction for a pair of dual linear programming problems

For a pair of dual (possibly improper) linear programming problems, a family of matrix corrections is studied that ensure the existence of given solutions to these problems. The case of correcting the coefficient matrix and three cases of correcting an augmented coefficient matrix (obtained by adding the right-hand side vector of the primal problem, the right-hand-side vector of the dual problem, or both vectors) are considered. Necessary and sufficient conditions for the existence of a solution to the indicated problems, its uniqueness is proved, and the form of matrices for the solution with a minimum Euclidean norm is presented. Numerical examples are given.     

On mixed and componentwise condition numbers for Moore-Penrose inverse and linear least squares problems

Classical condition numbers are normwise: they measure the size of both input perturbations and output errors using some norms. To take into account the relative of each data component, and, in particular, a possible data sparseness, componentwise condition numbers have been increasingly considered. These are mostly of two kinds: mixed and componentwise. In this paper, we give explicit expressions, computable from the data, for the mixed and componentwise condition numbers for the computation of the Moore-Penrose inverse as well as for the computation of solutions and residues of linear least squares problems. In both cases the data matrices have full column (row) rank.

     

Structured mixed and componentwise condition numbers of some structured matrices

Structured matrices, such as Cauchy, Vandermonde, Toeplitz, Hankel, and circulant matrices, are considered in this paper. We apply a Kronecker product-based technique to deduce the structured mixed and componentwise condition numbers for the matrix inversion and for the corresponding linear systems.     

Iterative refinement enhances the stability ofQR factorization methods for solving linear equations

Iterative refinement is a well-known technique for improving the quality of an approximate solution to a linear system. In the traditional usage residuals are computed in extended precision, but more recent work has shown that fixed precision is sufficient to yield benefits for stability. We extend existing results to show that fixed precision iterative refinement renders anarbitrary linear equations solver backward stable in a strong, componentwise sense, under suitable assumptions. Two particular applications involving theQR factorization are discussed in detail: solution of square linear systems and solution of least squares problems. In the former case we show that one step of iterative refinement suffices to produce a small componentwise relative backward error. Our results are weaker for the least squares problem, but again we find that iterative refinement improves a componentwise measure of backward stability. In particular, iterative refinement mitigates the effect of poor row scaling of the coefficient matrix, and so provides an alternative to the use of row interchanges in the HouseholderQR factorization. A further application of the results is described to fast methods for solving Vandermonde-like systems.     

Least squares solution with the minimum-norm to general matrix equations via iteration

Two iterative algorithms are presented in this paper to solve the minimal norm least squares solution to a general linear matrix equations including the well-known Sylvester matrix equation and Lyapunov matrix equation as special cases. The first algorithm is based on the gradient based searching principle and the other one can be viewed as its dual form. Necessary and sufficient conditions for the step sizes in these two algorithms are proposed to guarantee the convergence of the algorithms for arbitrary initial conditions. Sufficient condition that is easy to compute is also given. Moreover, two methods are proposed to choose the optimal step sizes such that the convergence speeds of the algorithms are maximized. Between these two methods, the first one is to minimize the spectral radius of the iteration matrix and explicit expression for the optimal step size is obtained. The second method is to minimize the square sum of the F-norm of the error matrices produced by the algorithm and it is shown that the optimal step size exits uniquely and lies in an interval. Several numerical examples are given to illustrate the efficiency of the proposed approach.     

四元数矩阵方程(AXB,CXD)=(E,F)的最小范数最小二乘Hermitian解

提出了研究四元数矩阵方程(AXB, CXD)=(E, F)的最小范数最小二乘Hermitian解的一个有效方法.首先应用四元数矩阵的实表示矩阵以及实表示矩阵的特殊结构,把四元数矩阵方程转化为相应的实矩阵方程,然后求出四元数矩阵方程(AXB, CXD)=(E, F)的最小二乘Hermitian解集,进而得到其最小范数最小二乘Hermitian解.所得到的结果只涉及实矩阵,相应的算法只涉及实运算,因此非常有效.最后的两个数值例子也说明了这一点.     

Perturbation analysis and condition numbers of scaled total least squares problems

The standard approaches to solving an overdetermined linear system Ax ≈ b find minimal corrections to the vector b and/or the matrix A such that the corrected system is consistent, such as the least squares (LS), the data least squares (DLS) and the total least squares (TLS). The scaled total least squares (STLS) method unifies the LS, DLS and TLS methods. The classical normwise condition numbers for the LS problem have been widely studied. However, there are no such similar results for the TLS and the STLS problems. In this paper, we first present a perturbation analysis of the STLS problem, which is a generalization of the TLS problem, and give a normwise condition number for the STLS problem. Different from normwise condition numbers, which measure the sizes of both input perturbations and output errors using some norms, componentwise condition numbers take into account the relation of each data component, and possible data sparsity. Then in this paper we give explicit expressions for the estimates of the mixed and componentwise condition numbers for the STLS problem. Since the TLS problem is a special case of the STLS problem, the condition numbers for the TLS problem follow immediately from our STLS results. All the discussions in this paper are under the Golub-Van Loan condition for the existence and uniqueness of the STLS solution. Yimin Wei is supported by the National Natural Science Foundation of China under grant 10871051, Shanghai Science & Technology Committee under grant 08DZ2271900 and Shanghai Education Committee under grant 08SG01. Sanzheng Qiao is partially supported by Shanghai Key Laboratory of Contemporary Applied Mathematics of Fudan University during his visiting.     

Indefinite Least Squares Problem with Quadratic Constraint and Its Condition Numbers

In this paper,we consider the indefinite least squares problem with quadratic constraint and its condition numbers.The conditions under which the problem has the unique solution are first presented.Then,the normwise,mixed,and componentwise condition numbers for solution and residual of this problem are derived.Numerical example is also provided to illustrate these results.     

Structured condition numbers and statistical condition estimation for the LDU factorization

In this article, we consider the structured condition numbers for LDU, factorization by using the modified matrix-vector approach and the differential calculus, which can be represented by sets of parameters. By setting the specific norms and weight parameters, we present the expressions of the structured normwise, mixed, componentwise condition numbers and the corresponding results for unstructured ones. In addition, we investigate the statistical estimation of condition numbers of LDU factorization using the probabilistic spectral norm estimator and the small-sample statistical condition estimation method, and devise three algorithms. Finally, we compare the structured condition numbers with the corresponding unstructured ones in numerical experiments.     

Condition numbers and perturbation analysis for the Tikhonov regularization of discrete ill‐posed problems

One of the most successful methods for solving the least‐squares problem min_x∥Ax?b∥₂ with a highly ill‐conditioned or rank deficient coefficient matrix A is the method of Tikhonov regularization. In this paper, we derive the normwise, mixed and componentwise condition numbers and componentwise perturbation bounds for the Tikhonov regularization. Our results are sharper than the known results. Some numerical examples are given to illustrate our results. Copyright © 2010 John Wiley & Sons, Ltd.     

Forward error analysis of Gaussian elimination

Summary Part I of this work deals with the forward error analysis of Gaussian elimination for general linear algebraic systems. The error analysis is based on a linearization method which determines first order approximations of the absolute errors exactly. Superposition and cancellation of error effects, structure and sparsity of the coefficient matrices are completely taken into account by this method. The most important results of the paper are new condition numbers and associated optimal component-wise error and residual estimates for the solutions of linear algebraic systems under data perturbations and perturbations by rounding erros in the arithmetic floating-point operations. The estimates do not use vector or matrix norms. The relative data and rounding condition numbers as well as the associated backward and residual stability constants are scaling-invariant. The condition numbers can be computed approximately from the input data, the intermediate results, and the solution of the linear system. Numerical examples show that by these means realistic bounds of the errors and the residuals of approximate solutions can be obtained. Using the forward error analysis, also typical results of backward error analysis are deduced. Stability theorems and a priori error estimates for special classes of linear systems are proved in Part II of this work.     

On the Complexity of Computing Error Bounds

We consider the cost of estimating an error bound for the computed solution of a system of linear equations, i.e., estimating the norm of a matrix inverse. Under some technical assumptions we show that computing even a coarse error bound for the solution of a triangular system of equations costs at least as much as testing whether the product of two matrices is zero. The complexity of the latter problem is in turn conjectured to be the same as matrix multiplication, matrix inversion, etc. Since most error bounds in practical use have much lower complexity, this means they should sometimes exhibit large errors. In particular, it is shown that condition estimators that: (1) perform at least one operation on each matrix entry; and (2) are asymptotically faster than any zero tester, must sometimes over or underestimate the inverse norm by a factor of at least , where n is the dimension of the input matrix, k is the bitsize, and where either or grows faster than any polynomial in n . Our results hold for the RAM model with bit complexity, as well as computations over rational and algebraic numbers, but not real or complex numbers. Our results also extend to estimating error bounds or condition numbers for other linear algebra problems such as computing eigenvectors. September 10, 1999. Final version received: August 23, 2000.     

矩阵方程AXA^T＋BYB^T=C的双对称最小二乘解及其最佳逼近

基于共轭梯度法的思想,通过特殊的变形,建立了一类求矩阵方程AXA^T＋BYB^T=C的双对称最小二乘解的迭代算法．对任意的初始双对称矩阵．在没有舍人误差的情况下,经过有限步迭代得到它的双对称最小二乘解;在选取特殊的初始双对称矩阵时,能得到它的的极小范数双对称最小二乘解．另外,给定任意矩阵,利用此方法可得到它的最佳逼近双对称解,数值例子表明,这种方法是有效的．