TLS和LS问题的比较

There are a number of articles discussing the total least squares(TLS) and the least squares(LS) problems.M.Wei(M.Wei, Mathematica Numerica Sinica 20(3)(1998),267-278) proposed a new orthogonal projection method to improve existing perturbation bounds of the TLS and LS problems.In this paper,wecontinue to improve existing bounds of differences between the squared residuals,the weighted squared residuals and the minimum norm correction matrices of the TLS and LS problems.     

关于TLS问题

1.引 言考虑观测线性系统AX=B,（1.1a）其中A∈Cm×n,B∈Cm×d（本文通篇假设m≥n d）,分别是精确但不可观测的A0∈Cm×n,B0∈Cm×d的近似,即精确线性系统是A0X=B0.（1.1b）Golub和Van Loan于1980年提出的总体最小二乘问题（以下简称TLS问题）就是求解线性系统AX=B（1.2）     

关于广义线性最小二乘问题的扰动分析

1问题 在应用统计中，常用的参数估计方法之一是广义线性最小二乘min(Cx-y)~TW~+(Cx-y).(1.1)其中C为m×n矩阵，W为m×m对称半正定矩阵，上标+代表Moore-Penrose广义逆Paige~[1]注意到：从统计观点看，W一般未必可逆，且通常具有对称满秩分解W=BB~T，因而，把问题改述为下述形式更合适     

A SUCCESSIVE LEAST SQUARES METHOD FOR STRUCTURED TOTAL LEAST SQUARES

A new method for Total Least Squares (TLS) problems is presented. It differs from previous approaches and is based on the solution of successive Least Squares problems.The method is quite suitable for Structured TLS (STLS) problems. We study mostly the case of Toeplitz matrices in this paper. The numerical tests illustrate that the method converges to the solution fast for Toeplitz STLS problems. Since the method is designed for general TLS problems, other structured problems can be treated similarly.     

关于TLS和LS解的扰动分析

1．引言本文采用卜］的记号．最小二乘（LS）和总体最小二乘（TLS）是科学计算中的两种重要方法．尤是TLS，近来已有多篇论文讨论［1－6，8－16］．奇异值分解（SVD）和CS分解是研究TLS和LS的重要工具．令ACm，BCm,C=(A,B)，A和C的SVD分别为（1．1）（1．2）其中P51为某个正整数，U，U，V，V均为西矩阵，UI，UI，VI，VI为上述矩阵的前P列，z1一山。g（。1，…，内），】2＝di。g（内十l，…，。小】1＝dl。g（61；…，站，】2二diag（4＋1；…，dk），。l三··2。120和dl三…三d。20分别为C和A的奇异值，Z＝mhfm．n十以…     

范德蒙型方程组最小二乘解的快速算法

在用多项式进行曲线拟合等实际问题中,需要求解以范德蒙型矩阵VT为系数阵的线性方程组VTx=b的最小二乘解.     

DIRECT ITERATIVE METHODS FOR RANK DEFICIENT GENERALIZED LEAST SQUARES PROBLEMS

1. IntroductionThe generalized LS problemis frequently found in solving problems from statistics, engineering, economics, imageand signal processing. Here A e Rmxn with m 2 n, b E Re and W E Rmxm issymmetric positive definite. The large sparse rank deficient generalized LS problemsappeal in computational genetics when we consider mited linear model for tree oranimal genetics [2], [31, [5].Recentlyg Yuan [9] and [10], Yuan and lusem [11] considered direct iterative methodsfor the problem …     

线性子空间上求解AX=B的最小二乘问题的迭代算法

应用共轭梯度方法和线性投影算子,给出迭代算法求解了线性矩阵方程AX=B在任意线性子空间上的最小二乘解问题.在不考虑舍入误差的情况下,可以证明,所给迭代算法经过有限步迭代可得到矩阵方程AX=B的最小二乘解、极小范数最小二乘解及其最佳逼近.文中的数值例子证实了该算法的有效性.     

Perturbation analysis and condition numbers of scaled total least squares problems

The standard approaches to solving an overdetermined linear system Ax ≈ b find minimal corrections to the vector b and/or the matrix A such that the corrected system is consistent, such as the least squares (LS), the data least squares (DLS) and the total least squares (TLS). The scaled total least squares (STLS) method unifies the LS, DLS and TLS methods. The classical normwise condition numbers for the LS problem have been widely studied. However, there are no such similar results for the TLS and the STLS problems. In this paper, we first present a perturbation analysis of the STLS problem, which is a generalization of the TLS problem, and give a normwise condition number for the STLS problem. Different from normwise condition numbers, which measure the sizes of both input perturbations and output errors using some norms, componentwise condition numbers take into account the relation of each data component, and possible data sparsity. Then in this paper we give explicit expressions for the estimates of the mixed and componentwise condition numbers for the STLS problem. Since the TLS problem is a special case of the STLS problem, the condition numbers for the TLS problem follow immediately from our STLS results. All the discussions in this paper are under the Golub-Van Loan condition for the existence and uniqueness of the STLS solution. Yimin Wei is supported by the National Natural Science Foundation of China under grant 10871051, Shanghai Science & Technology Committee under grant 08DZ2271900 and Shanghai Education Committee under grant 08SG01. Sanzheng Qiao is partially supported by Shanghai Key Laboratory of Contemporary Applied Mathematics of Fudan University during his visiting.     

关于TLS的可解性及扰动分析

尽管有关总体最小二乘问题的研究工作是大量的，然而ＴＬＳ可解的充分必要条件一直没有得到。本文首先给出完整的可解性分析，然后建立了ＴＬＳ的扰动上界。     

加权总体最小二乘问题的分析

总体最小二乘问题由Golub和Van Loan首先进行数学的分析,随后人们对于总体最小二乘问题的算法、解的各种形式、总体最小二乘解和最小二乘解的关系、总体最小二乘解的扰动理论以及数值试验作了大量的研究工作。近来,[10]中给出了总体最小二乘问题(TLS)较一般地讨论。另一方面,Golub和Van Loan研究了总体最小二乘问题的特殊均加权形式。本文试图在[10,11]的基础上讨论最一般的总体最小二     

On greedy randomized coordinate descent methods for solving large linear least‐squares problems

For solving large scale linear least‐squares problem by iteration methods, we introduce an effective probability criterion for selecting the working columns from the coefficient matrix and construct a greedy randomized coordinate descent method. It is proved that this method converges to the unique solution of the linear least‐squares problem when its coefficient matrix is of full rank, with the number of rows being no less than the number of columns. Numerical results show that the greedy randomized coordinate descent method is more efficient than the randomized coordinate descent method.     

一类线性约束矩阵不等式及其最小二乘问题

本文主要考虑一类线性矩阵不等式及其最小二乘问题,它等价于相应的矩阵不等式最小非负偏差问题.之前相关文献提出了求解该类最小非负偏差问题的迭代方法,但该方法在每步迭代过程中需要精确求解一个约束最小二乘子问题,因此对规模较大的问题,整个迭代过程需要耗费巨大的计算量.为了提高计算效率,本文在现有算法的基础上,提出了一类修正迭代方法.该方法在每步迭代过程中利用有限步的矩阵型LSQR方法求解一个低维矩阵Krylov子空间上的约束最小二乘子问题,降低了整个迭代所需的计算量.进一步运用投影定理以及相关的矩阵分析方法证明了该修正算法的收敛性,最后通过数值例子验证了本文的理论结果以及算法的有效性.     

On condition numbers for Moore–Penrose inverse and linear least squares problem involving Kronecker products

In this paper, we investigate the normwise, mixed, and componentwise condition numbers and their upper bounds for the Moore–Penrose inverse of the Kronecker product and more general matrix function compositions involving Kronecker products. We also present the condition numbers and their upper bounds for the associated Kronecker product linear least squares solution with full column rank. In practice, the derived upper bounds for the mixed and componentwise condition numbers for Kronecker product linear least squares solution can be efficiently estimated using the Hager–Higham Algorithm. Copyright © 2012 John Wiley & Sons, Ltd.     

On the almost rank deficient case of the least squares problem

This paper studies properties of the solutions to overdetermined systems of linear equations whose matrices are almost rank deficient. Let such a system be approximated by the system of rankr which is closest in the euclidean matrix norm. The residual of the approximate solution depends on the scaling of the independent variable. Sharp bounds are given for the sensitivity of the residual to the scaling of the independent variable. It turns out that these bounds depend critically on a few factors which can be computed in connection with the singular value decomposition. Further the influence from the scaling on the pseudo-inverse solution of a rank deficient system is estimated.This work was sponsored by the Swedish Institute of Applied Mathematics.     

Bounds for the least squares distance using scaled total least squares

Summary. The standard approaches to solving overdetermined linear systems construct minimal corrections to the data to make the corrected system compatible. In ordinary least squares (LS) the correction is restricted to the right hand side c, while in scaled total least squares (STLS) [14,12] corrections to both c and B are allowed, and their relative sizes are determined by a real positive parameter . As , the STLS solution approaches the LS solution. Our paper [12] analyzed fundamentals of the STLS problem. This paper presents a theoretical analysis of the relationship between the sizes of the LS and STLS corrections (called the LS and STLS distances) in terms of . We give new upper and lower bounds on the LS distance in terms of the STLS distance, compare these to existing bounds, and examine the tightness of the new bounds. This work can be applied to the analysis of iterative methods which minimize the residual norm, and the generalized minimum residual method (GMRES) [15] is used here to illustrate our theory. Received July 20, 2000 / Revised version received February 28, 2001 / Published online July 25, 2001     

拟合模糊观测数据的线性回归模型

本文讨论了实验观测数据为一般模糊数的线性最优拟合问题，通过定义模糊数空间中的距离，建立了模糊数空间到模糊数空间的回归模型，证明了最小二乘问题的解与其正则方程组的解的一致性，进而由正则方程组导出了问题的显式解。本模型的计算简便，具有实用价值。     

加权总体最小二乘问题的解集和性质

本文讨论了加权总体最小二乘问题的等价解集，分析了加权总体最小二乘解与加权最小二乘问题的解之间的关系。推广了Ｇｏｌｕｂ和Ｖａｎ Ｌｏａｎ，Ｖａｎ Ｈｕｆｆｅｌ和Ｖａｎｄｅｗａｌｌｅ，及Ｗｅｉ的相应结果。     

矩阵方程ATXB+BTXTA=D的极小范数最小二乘解

1引言本文用Rm×n表示所有m×n实矩阵全体,ORn×n,ASRn×n分别表示n×n实正交矩阵类与反对称矩阵类.‖·‖F表示矩阵的Frobenius范数,A+为矩阵A的Moore-Penrose广义逆,A*B与A(?)B分别表示矩阵4与B的Hadamard乘积及Kronecker乘积,即若A=(aij),B=(bij),则A*B=(ajibij),A(?)B=(aijB),vec4表示矩阵A的按行拉直,即若A=[aT1,aT2,…,aTm],其中ai为A的行向量,则vecA=(a1a2…am)T.设A∈Rn×m,B∈Rp×m,D∈Rm×m,我们考虑不相容线性矩阵方程ATXB+BTXTA=D(1.1)     

矩阵方程AXB+CY D=E的三对角中心对称极小范数最小二乘解

本文研究了矩阵方程AXB+CY D=E的三对角中心对称极小范数最小二乘解问题.利用矩阵的Kronecker积和Moore-Penrose广义逆方法,得到了矩阵方程AXB+CY D=E的三对角中心对称极小范数最小二乘解的表达式.