Least‐squares solutions and least‐rank solutions of the matrix equation AXA* = B and their relations

A Hermitian matrix X is called a least‐squares solution of the inconsistent matrix equation AXA^* = B, where B is Hermitian. A^* denotes the conjugate transpose of A if it minimizes the F‐norm of B ? AXA^*; it is called a least‐rank solution of AXA^* = B if it minimizes the rank of B ? AXA^*. In this paper, we study these two types of solutions by using generalized inverses of matrices and some matrix decompositions. In particular, we derive necessary and sufficient conditions for the two types of solutions to coincide. Copyright © 2012 John Wiley & Sons, Ltd.     

Perturbation analysis and condition numbers of scaled total least squares problems

The standard approaches to solving an overdetermined linear system Ax ≈ b find minimal corrections to the vector b and/or the matrix A such that the corrected system is consistent, such as the least squares (LS), the data least squares (DLS) and the total least squares (TLS). The scaled total least squares (STLS) method unifies the LS, DLS and TLS methods. The classical normwise condition numbers for the LS problem have been widely studied. However, there are no such similar results for the TLS and the STLS problems. In this paper, we first present a perturbation analysis of the STLS problem, which is a generalization of the TLS problem, and give a normwise condition number for the STLS problem. Different from normwise condition numbers, which measure the sizes of both input perturbations and output errors using some norms, componentwise condition numbers take into account the relation of each data component, and possible data sparsity. Then in this paper we give explicit expressions for the estimates of the mixed and componentwise condition numbers for the STLS problem. Since the TLS problem is a special case of the STLS problem, the condition numbers for the TLS problem follow immediately from our STLS results. All the discussions in this paper are under the Golub-Van Loan condition for the existence and uniqueness of the STLS solution. Yimin Wei is supported by the National Natural Science Foundation of China under grant 10871051, Shanghai Science & Technology Committee under grant 08DZ2271900 and Shanghai Education Committee under grant 08SG01. Sanzheng Qiao is partially supported by Shanghai Key Laboratory of Contemporary Applied Mathematics of Fudan University during his visiting.     

关于TLS和LS解的扰动分析

1．引言本文采用卜］的记号．最小二乘（LS）和总体最小二乘（TLS）是科学计算中的两种重要方法．尤是TLS，近来已有多篇论文讨论［1－6，8－16］．奇异值分解（SVD）和CS分解是研究TLS和LS的重要工具．令ACm，BCm,C=(A,B)，A和C的SVD分别为（1．1）（1．2）其中P51为某个正整数，U，U，V，V均为西矩阵，UI，UI，VI，VI为上述矩阵的前P列，z1一山。g（。1，…，内），】2＝di。g（内十l，…，。小】1＝dl。g（61；…，站，】2二diag（4＋1；…，dk），。l三··2。120和dl三…三d。20分别为C和A的奇异值，Z＝mhfm．n十以…     

Preconditioned conjugate gradient method for generalized least squares problems

A variant of the preconditioned conjugate gradient method to solve generalized least squares problems is presented. If the problem is min (Ax − b)^TW⁻¹(Ax − b) with A ∈ R^m×n and W ∈ R^m×m symmetric and positive definite, the method needs only a preconditioner A₁ ∈ R^n×n, but not the inverse of matrix W or of any of its submatrices. Freund's comparison result for regular least squares problems is extended to generalized least squares problems. An error bound is also given.     

The least squares g-inverses for sum of matrices

In this article, we exhibit under suitable conditions a neat relationship between the least squares g-inverse for a sum of two matrices and the least squares g-inverses of the individual terms. We give a necessary and sufficient condition for the set equations (A?+?B){1,?3}?=?A{1,?3}?+?B{1,?3} and (A?+?B){1,?4}?=?A{1,?4}?+?B{1,?4}.     

A review of existence criteria for parameter estimation of the Michaelis–Menten regression model

In this paper we consider the least squares (LS) and total least squares (TLS) problems for a Michaelis–Menten enzyme kinetic model f(x; a, b) = ax/(b + x), a, b > 0. In various applied research such as biochemistry, pharmacology, biology and medicine there are lots of different applications of this model. We will systematize some of our results pertaining to the existence of the LS and TLS estimate, which were proved in Hadeler et al. (Math Method Appl Sci 30:1231–1241, 2007) and Jukić et al. (J Comput Appl Math 201:230–246, 2007). Finally, we suggest a choice of good initial approximation and give one numerical example.      

TLS和LS问题的比较

There are a number of articles discussing the total least squares(TLS) and the least squares(LS) problems.M.Wei(M.Wei, Mathematica Numerica Sinica 20(3)(1998),267-278) proposed a new orthogonal projection method to improve existing perturbation bounds of the TLS and LS problems.In this paper,wecontinue to improve existing bounds of differences between the squared residuals,the weighted squared residuals and the minimum norm correction matrices of the TLS and LS problems.     

ON THE ACCURACY OF THE LEAST SQUARES AND THE TOTAL LEAST SQUARES METHODS

Consider solving an overdetermined system of linear algebraic equations by both the least squares method (LS) and the total least squares method (TLS). Extensive published computational evidence shows that when the original system is consistent. one often obtains more accurate solutions by using the TLS method rather than the LS method. These numerical observations contrast with existing analytic perturbation theories for the LS and TLS methods which show that the upper bounds for the LS solution are always smaller than the corresponding upper bounds for the TLS solutions. In this paper we derive a new upper bound for the TLS solution and indicate when the TLS method can be more accurate than the LS method.Many applied problems in signal processing lead to overdetermined systems of linear equations where the matrix and right hand side are determined by the experimental observations (usually in the form of a lime series). It often happens that as the number of columns of the matrix becomes larger, the ra     

Algebraic connections between the least squares and total least squares problems

Summary In this paper the closeness of the total least squares (TLS) and the classical least squares (LS) problem is studied algebraically. Interesting algebraic connections between their solutions, their residuals, their corrections applied to data fitting and their approximate subspaces are proven.All these relationships point out the parameters which mainly determine the equivalences and differences between the two techniques. These parameters also lead to a better understanding of the differences in sensitivity between both approaches with respect to perturbations of the data.In particular, it is shown how the differences between both approaches increase when the equationsAXB become less compatible, when the length ofB orX is growing or whenA tends to be rank-deficient. They are maximal whenB is parallel with the singular vector ofA associated with its smallest singular value. Furthermore, it is shown how TLS leads to a weighted LS problem, and assumptions about the underlying perturbation model of both techniques are deduced. It is shown that many perturbation models correspond with the same TLS solution.Senior Research Assistant of the Belgian N.F.W.O. (National Fund of Scientific Research)     

The least squares problem of the matrix equation A1X1B1^T ＋ A2X2B2^T = T

The matrix least squares （LS） problem minx ｜｜AXB^T--T｜｜F is trivial and its solution can be simply formulated in terms of the generalized inverse of A and B. Its generalized problem minx1,x2 ｜｜A1X1B1^T ＋ A2X2B2^T - T｜｜F can also be regarded as the constrained LS problem minx=diag（x1,x2） ｜｜AXB^T -T｜｜F with A = [A1, A2] and B = [B1, B2]. The authors transform T to T such that min x1,x2 ｜｜A1X1B1^T＋A2X2B2^T -T｜｜F is equivalent to min x=diag（x1 ,x2） ｜｜AXB^T - T｜｜F whose solutions are included in the solution set of unconstrained problem minx ｜｜AXB^T - T｜｜F. So the general solutions of min x1,x2 ｜｜A1X1B^T ＋ A2X2B2^T -T｜｜F are reconstructed by selecting the parameter matrix in that of minx ｜｜AXB^T - T｜｜F.     

A QR Decomposition for Matrix Pencils

This paper describes an efficient and numerically stable modification of the QR decomposition for solving a parametric set of linear least squares problems with a parametric matrix A + B for several values of the parameter . The method is demonstrated on a typical application.     

Backward perturbation analysis for scaled total least‐squares problems

The scaled total least‐squares (STLS) method unifies the ordinary least‐squares (OLS), the total least‐squares (TLS), and the data least‐squares (DLS) methods. In this paper we perform a backward perturbation analysis of the STLS problem. This also unifies the backward perturbation analyses of the OLS, TLS and DLS problems. We derive an expression for an extended minimal backward error of the STLS problem. This is an asymptotically tight lower bound on the true minimal backward error. If the given approximate solution is close enough to the true STLS solution (as is the goal in practice), then the extended minimal backward error is in fact the minimal backward error. Since the extended minimal backward error is expensive to compute directly, we present a lower bound on it as well as an asymptotic estimate for it, both of which can be computed or estimated more efficiently. Our numerical examples suggest that the lower bound gives good order of magnitude approximations, while the asymptotic estimate is an excellent estimate. We show how to use our results to easily obtain the corresponding results for the OLS and DLS problems in the literature. Copyright © 2009 John Wiley & Sons, Ltd.     

Some extensions of the Kantorovich inequality and statistical applications

Kantorovich gave an upper bound to the product of two quadratic forms, (X′AX) (X′A⁻¹X), where X is an n-vector of unit length and A is a positive definite matrix. Bloomfield, Watson and Knott found the bound for the product of determinants |X′AX| |X′A⁻¹X| where X is n × k matrix such that X′X = I_k. In this paper we determine the bounds for the traces and determinants of matrices of the type X′AYY′A⁻¹X, X′B²X(X′BCX)⁻¹ X′C²X(X′BCX)⁻¹ where X and Y are n × k matrices such that X′X = Y′Y = I_k and A, B, C are given matrices satisfying some conditions. The results are applied to the least squares theory of estimation.     

A Faster and Simpler Recursive Algorithm for the LAPACK Routine DGELS

We present new algorithms for computing the linear least squares solution to overdetermined linear systems and the minimum norm solution to underdetermined linear systems. For both problems, we consider the standard formulation min AX – B _F and the transposed formulation min A ^T X – B _F , i.e, four different problems in all. The functionality of our implementation corresponds to that of the LAPACK routine DGELS. The new implementation is significantly faster and simpler. It outperforms the LAPACK DGELS for all matrix sizes tested. The improvement is usually 50–100% and it is as high as 400%. The four different problems of DGELS are essentially reduced to two, by use of explicit transposition of A. By explicit transposition we avoid computing Householder transformations on vectors with large stride. The QR factorization of block columns of A is performed using a recursive level-3 algorithm. By interleaving updates of B with the factorization of A, we reduce the number of floating point operations performed for the linear least squares problem. By avoiding redundant computations in the update of B we reduce the work needed to compute the minimum norm solution. Finally, we outline fully recursive algorithms for the four problems of DGELS as well as for QR factorization.This revised version was published online in October 2005 with corrections to the Cover Date.     

Algebraic relations between the total least squares and least squares problems with more than one solution

Summary This paper completes our previous discussion on the total least squares (TLS) and the least squares (LS) problems for the linear systemAX=B which may contain more than one solution [12, 13], generalizes the work of Golub and Van Loan [1,2], Van Huffel [8], Van Huffel and Vandewalle [11]. The TLS problem is extended to the more general case. The sets of the solutions and the squared residuals for the TLS and LS problems are compared. The concept of the weighted squares residuals is extended and the difference between the TLS and the LS approaches is derived. The connection between the approximate subspaces and the perturbation theories are studied.It is proved that under moderate conditions, all the corresponding quantities for the solution sets of the TLS and the modified LS problems are close to each other, while the quantities for the solution set of the LS problem are close to the corresponding ones of a subset of that of the TLS problem.This work was financially supported by the Education Committee, People's Republic of China     

Matrix stretching for sparse least squares problems

For linear least squares problems min _x ‖Ax − b‖₂, where A is sparse except for a few dense rows, a straightforward application of Cholesky or QR factorization will lead to catastrophic fill in the factor R. We consider handling such problems by a matrix stretching technique, where the dense rows are split into several more sparse rows. We develop both a recursive binary splitting algorithm and a more general splitting method. We show that for both schemes the stretched problem has the same set of solutions as the original least squares problem. Further, the condition number of the stretched problem differs from that of the original by only a modest factor, and hence the approach is numerically stable. Experimental results from applying the recursive binary scheme to a set of modified matrices from the Harwell‐Boeing collection are given. We conclude that when A has a small number of dense rows relative to its dimension, there is a significant gain in sparsity of the factor R. A crude estimate of the optimal number of splits is obtained by analysing a simple model problem. Copyright © 2000 John Wiley & Sons, Ltd.     

Fast enclosure for the minimum norm least squares solution of the matrix equation AXB = C

Fast algorithms for enclosing the minimum norm least squares solution of the matrix equation AXB = C are proposed. To develop these algorithms, theories for obtaining error bounds of numerical solutions are established. The error bounds obtained by these algorithms are verified in the sense that all the possible rounding errors have been taken into account. Techniques for accelerating the enclosure and obtaining smaller error bounds are introduced. Numerical results show the properties of the proposed algorithms. Copyright © 2015 John Wiley & Sons, Ltd.     

On the solutions of the equation AXB = C under Toeplitz‐like and Hankel matrices constraint

In this paper, we are mainly concerned with 2 types of constrained matrix equation problems of the form AXB=C, the least squares problem and the optimal approximation problem, and we consider several constraint matrices, such as general Toeplitz matrices, upper triangular Toeplitz matrices, lower triangular Toeplitz matrices, symmetric Toeplitz matrices, and Hankel matrices. In the first problem, owing to the special structure of the constraint matrix , we construct special algorithms; necessary and sufficient conditions are obtained about the existence and uniqueness for the solutions. In the second problem, we use von Neumann alternating projection algorithm to obtain the solutions of problem. Then we give 2 numerical examples to demonstrate the effectiveness of the algorithms.     

Hereditary invertible linear surjections and splitting problems for selections

Let A+B be the pointwise (Minkowski) sum of two convex subsets A and B of a Banach space. Is it true that every continuous mapping h:X→A+B splits into a sum h=f+g of continuous mappings f:X→A and g:X→B? We study this question within a wider framework of splitting techniques of continuous selections. Existence of splittings is guaranteed by hereditary invertibility of linear surjections between Banach spaces. Some affirmative and negative results on such invertibility with respect to an appropriate class of convex compacta are presented. As a corollary, a positive answer to the above question is obtained for strictly convex finite-dimensional precompact spaces.     

Least squares X=±Xη* solutions to split quaternion matrix equation AXAη*=B

In the paper, the split quaternion matrix equation AXA^η*=B is considered, where the operator A^η* is the η-conjugate transpose of A, where η∈{i,j,k}. We propose some new real representations, which well exploited the special structures of the original matrices. By using this method, we obtain the necessary and sufficient conditions for AXA^η*=B to have X=±X^η* solutions and derive the general expressions of solutions when it is consistent. In addition, we also derive the general expressions of the least squares X=±X^η* solutions to it in case that this matrix equation is not consistent.