等式约束加权线性最小二乘问题的解法

1 引言 在实际应用中常会提出解等式约束加权线性最小二乘问题 min||b-Ax||_M,(1.1) x∈C~n s.t.Bx=d, 其中B∈C~(p×n),A∈C~(q×n),d∈C~p,b∈C~q,M∈C~(q×q)为Hermite正定阵. 对于问题(1.1),目前已有多种解法,见文[1—3).本文将利用广义逆矩阵的知识,给出(1.1)的通解及迭代解法.本文中关于矩阵广义逆与投影算子(矩阵)的记号基本上与文[4]的相同.例如,A~+表示A的MP逆,P_L表示到子空间L上的正交投影算子,λ_(max)(MAY)表示矩阵M~(1/2)AY的最大特征值.我们还要用到广义BD逆的概念: 设A∈C~(n×n),L为C~n的子空间,则称A_(L)~(+)=P_L(AP_L+P_L⊥)~+为A关于L的广义BD逆.     

解等式约束加权线性最小二乘问题的矩阵校正方法

1 引言 在实际应用中常会提出解等式约束加权线性最小二乘问题 min(b_2-A_2x)~TW(b_2-A_2x) x∈R~n (1) s.t.A_1x=b_1,其中A_1∈R~(p×n),A~2∈R(q×n),b_1∈R~p,b_2∈R~q,W∈R(q×q)为对称正定矩阵. 对于问题(1),目前已有多种数值求解方法,如Paige利用(1)的对偶公式给出了一个向后稳定的数值方法.Gulliksson和Wedin利用加权QR分解技巧给出了解(1)的一个直接解法.作者利用广义Cholesky分解构造了解(1)的矩阵分解方法.     

不定最小二乘问题的改进的不完全双曲Gram-Schmidt预处理算法

应用改进的不完全双曲Gram-Schmidt(IHMGS)方法预处理不定最小二乘问题的共轭梯度法(CGILS)、正交分解法(ILSQR)与广义的最小剩余法(GMRES)等迭代算法来求解大型稀疏的不定最小二乘问题.数值实验表明,IHMGS预处理方法可有效提高相应算法的迭代速度,且当矩阵的条件数比较大时,效果更加显著.     

加权广义逆、加权最小二乘和约束最小二乘问题

本文采用如下记号:记C~m×n是具有复数域的m×n长方矩阵的集合,C~m=C~m×1是m维向量的集合.对A∈C~m×n称A~H∈C~m×n是A的共轭转置矩阵,rank(A)表示A的秩,R(A)和N(A)分别为A的值域和零空间,||·||=||·||2和||·||F分别为2-范数和Frobenius范数;I表示恒等矩阵.人们在研究数学规划、数值分析、数据处理,散射理论和电磁学等领域中都将问题归纳为如下的最小二乘问题:     

解非线性最小二乘问题的锥模型算法

在自然科学研究、经济、统计等领域,非线性最小二乘有着广泛的应用,因而寻找快捷有效的算法有着十分重要的意义。它首先是一个最优化问题,同时又有自身的结构特点,充分利用其结构特点,是寻找更有效算法的关键。     

具二次等式约束最小二乘问题的一种极端情形

本文研究具有二次等式约束的最小二乘问题(LSS): min‖Ax-b‖₂ s.t. ‖x‖₂=1, 其中A∈R^m×n, b∈R^m, 并假定‖A⁺b‖₂<1.重点关注一个极端情形: ‖A⁺b‖₂≈0. 敏度分析表明,这是一种病态问题. 基于Padé逼近给出了一种迭代解法. 数值算例表明,新方法在速度上较已有方法有优势.     

关于求解带约束秩亏线性回归方程组最小二乘解的一个算法

1 引 言 考虑带线性约束的秩亏线性回归模型: Y=Xβ+ε, Hβ=c, (M_1) ε～N(0,σ~2V), V≥0,及带线性约束和非负性约束的秩亏线性回归模型:     

基于约束总体最小二乘方法的近似消逝理想算法

提出了基于约束总体最小二乘方法的近似消逝理想算法.给定经验点集Xε,该算法输出序理想(O)和多项式集合(g).当(O)中单项的个数等于经验点集Xε的基数时,(g)即为Xε的近似消逝理想基.该算法充分考虑赋值向量的扰动之间的内在联系,因此在关注向量的数值相关性方面,算法优于目前其它同类算法.     

模糊线性回归模型的约束最小二乘估计

自Tanaka等1982年提出模糊回归概念以来,该问题已得到广泛的研究。作为主要估计方法之一的模糊最小二乘估计以其与统计最小二乘估计的密切联系更受到人们的重视。本文依据适当定义的两个模糊数之间的距离,提出了模糊线性回归模型的一个约束最小二乘估计方法,该方法不仅能使估计的模糊参数的宽度具有非负性而且估计的模糊参数的中心线与传统的最小二乘估计相一致。最后,通过数值例子说明了所提方法的具体应用。     

Real structure-preserving algorithm for quaternion equality constrained least squares problem

Quaternion equality constrained least squares problem is an extremely effective tool in studying quantum mechanics and quantum field theory. However, the computation of the quaternion equality constrained least squares problem is extremely complex. In this paper, we first prove that quaternion equality constrained least squares problem is equivalent to weighted quaternion least squares problem when the parameter $\tau \to +\infty$. Then, for weighted quaternion least squares problem, applying the special structure of real representation of quaternion, we propose real structure–preserving algorithm to obtain the solution of quaternion equality contained least squares problem. At last, we give numerical examples to illustrate the effectiveness of our method.     

On the modified Gram-Schmidt algorithm for weighted and constrained linear least squares problems

A framework and an algorithm for using modified Gram-Schmidt for constrained and weighted linear least squares problems is presented. It is shown that a direct implementation of a weighted modified Gram-Schmidt algorithm is unstable for heavily weighted problems. It is shown that, in most cases it is possible to get a stable algorithm by a simple modification free from any extra computational costs. In particular, it is not necessary to perform reorthogonalization.Solving the weighted and constrained linear least squares problem with the presented weighted modified Gram-Schmidt algorithm is seen to be numerically equivalent to an algorithm based on a weighted Householder-likeQR factorization applied to a slightly larger problem. This equivalence is used to explain the instability of the weighted modified Gram-Schmidt algorithm. If orthogonality, with respect to a weighted inner product, of the columns inQ is important then reorthogonalization can be used. One way of performing such reorthogonalization is described.Computational tests are given to show the main features of the algorithm.     

A new technique of quaternion equality constrained least squares problem

By means of complex representation of a quaternion matrix, we study the relationship between the solutions of the quaternion equality constrained least squares problem and that of complex equality constrained least squares problem, and obtain a new technique of finding a solution of the quaternion equality constrained least squares problem.     

Some new properties of the equality constrained and weighted least squares problem

In this paper, some new properties of the equality constrained and weighted least squares problem (WLSE) min W^1/2(Kx−g)₂ subject to Lx=h are obtained. We derive a perturbation bound based on an unconstrained least squares problem and deduce some equivalent formulae for the projectors of this unconstrained LS problem. We also present a new way to compute the minimum norm solution x_WLSE of the WLSE problem by using the QR decomposition of the corresponding matrices and propose an algorithm to compute x_WLSE using the QR factorizations. Some numerical examples are provided to compare different methods for solving the WLSE problem.     

Algebraic methods for equality constrained least squares problems in commutative quaternionic theory

This paper, by means of two matrix representations of a commutative quaternion matrix, studies the relationship between the solutions of commutative quaternion equality constrained least squares (LSE) problems and that of complex and real LSE problems and derives two algebraic methods for finding the solutions of equality constrained least squares problems in commutative quaternionic theory.     

A continuation method for solving separable nonlinear least squares problems

Given the data (x_i, y_i), i=1, 2, …, n, the problem is to find the values of the linear and nonlinear parameters â and b? which minimize the nonlinear functional |F(b)a?y|₂² over $\text{a ?}\text{R}{}^{\mathrm{p}}\text{, b ?}\text{R}{}^{\mathrm{q}}\text{,}\text{where}\text{F ?}\text{R}{}^{\mathrm{n\times p}}$ is a variable matrix and assumed to be of full rank, and $\text{y ?}\text{R}{}^{\mathrm{n}}$ is a constant vector.In this paper, we present a method for solving this problem by imbedding it into a one-parameter family of problems and by following its solution path using a predictor-corrector algorithm. In the course of iterations, the original problem containing p+q+1 variables is transformed into a problem with q+1 nonlinear variables by taking the separable structure of the problem into account. By doing so, the method reduces to solving a series of equations of smaller size and a considerable saving in the storage is obtained.Results of numerical experiments are reported to demonstrate the effectiveness of the proposed method.     

A parallel multisplitting solution of the least squares problem

The linear least squares problem, min_x∥Ax − b∥₂, is solved by applying a multisplitting (MS) strategy in which the system matrix is decomposed by columns into p blocks. The b and x vectors are partitioned consistently with the matrix decomposition. The global least squares problem is then replaced by a sequence of local least squares problems which can be solved in parallel by MS. In MS the solutions to the local problems are recombined using weighting matrices to pick out the appropriate components of each subproblem solution. A new two-stage algorithm which optimizes the global update each iteration is also given. For this algorithm the updates are obtained by finding the optimal update with respect to the weights of the recombination. For the least squares problem presented, the global update optimization can also be formulated as a least squares problem of dimension p. Theoretical results are presented which prove the convergence of the iterations. Numerical results which detail the iteration behavior relative to subproblem size, convergence criteria and recombination techniques are given. The two-stage MS strategy is shown to be effective for near-separable problems. © 1998 John Wiley & Sons, Ltd.     

Model reduction for large-scale dynamical systems via equality constrained least squares

In this paper, we present a new method of model reduction for large-scale dynamical systems, which belongs to the SVD-Krylov based method category. It is a two-sided projection where one side reflects the Krylov part and the other side reflects the SVD (observability gramian) part. The reduced model matches the first r+i Markov parameters of the full order model, and the remaining ones approximate in a least squares sense without being explicitly computed, where r is the order of the reduced system, and i is a nonnegative integer such that 1≤i<r. The reduced system minimizes a weighted ?₂ error. By the definition of a shift operator, the proposed approximation is also obtained by solving an equality constrained least squares problem. Moreover, the method is generalized for moment matching at arbitrary interpolation points. Several numerical examples verify the effectiveness of the approach.