Weighted least squares solutions to general coupled Sylvester matrix equations

This paper is concerned with weighted least squares solutions to general coupled Sylvester matrix equations. Gradient based iterative algorithms are proposed to solve this problem. This type of iterative algorithm includes a wide class of iterative algorithms, and two special cases of them are studied in detail in this paper. Necessary and sufficient conditions guaranteeing the convergence of the proposed algorithms are presented. Sufficient conditions that are easy to compute are also given. The optimal step sizes such that the convergence rates of the algorithms, which are properly defined in this paper, are maximized and established. Several special cases of the weighted least squares problem, such as a least squares solution to the coupled Sylvester matrix equations problem, solutions to the general coupled Sylvester matrix equations problem, and a weighted least squares solution to the linear matrix equation problem are simultaneously solved. Several numerical examples are given to illustrate the effectiveness of the proposed algorithms.     

广义耦合Sylvester矩阵方程的对称-反对称最小二乘解

本文主要研究极小残差问题‖(A1XB1+C1YD1A2XB2+C2YD2)-(M1M2)‖=min关于X对称-Y反对称解的迭代算法.本文首先给出等价于极小残差问题的规范方程,然后,提出求解此规范方程的对称-反对称解的迭代算法.在不考虑舍入误差的情况下,任取一个初始的对称-反对称矩阵对(X0,Y0),该算法都可以在有限步内求得该极小残差问题的对称-反对称解.最后讨论该问题的极小范数对称-反对称解.     

Multipole fast algorithm for the least‐squares approach of the method of fundamental solutions for three‐dimensional harmonic problems

In this article we describe an improvement in the speed of computation for the least‐squares method of fundamental solutions (MFS) by means of Greengard and Rokhlin's FMA. Iterative solution of the linear system of equations is performed for the equations given by the least‐squares formulation of the MFS. The results of applying the method to test problems from potential theory with a number of boundary points in the order of 80,000 show that the method can achieve fast solutions for the potential and its directional derivatives. The results show little loss of accuracy and a major reduction in the memory requirements compared to the direct solution method of the least squares problem with storage of the full MFS matrix. The method can be extended to the solution of overdetermined systems of equations arising from boundary integral methods with a large number of boundary integration points. © 2003 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq 19: 828–845, 2003.     

线性子空间上求解AX=B的最小二乘问题的迭代算法

应用共轭梯度方法和线性投影算子,给出迭代算法求解了线性矩阵方程AX=B在任意线性子空间上的最小二乘解问题.在不考虑舍入误差的情况下,可以证明,所给迭代算法经过有限步迭代可得到矩阵方程AX=B的最小二乘解、极小范数最小二乘解及其最佳逼近.文中的数值例子证实了该算法的有效性.     

Least squares solution with the minimum-norm to general matrix equations via iteration

Two iterative algorithms are presented in this paper to solve the minimal norm least squares solution to a general linear matrix equations including the well-known Sylvester matrix equation and Lyapunov matrix equation as special cases. The first algorithm is based on the gradient based searching principle and the other one can be viewed as its dual form. Necessary and sufficient conditions for the step sizes in these two algorithms are proposed to guarantee the convergence of the algorithms for arbitrary initial conditions. Sufficient condition that is easy to compute is also given. Moreover, two methods are proposed to choose the optimal step sizes such that the convergence speeds of the algorithms are maximized. Between these two methods, the first one is to minimize the spectral radius of the iteration matrix and explicit expression for the optimal step size is obtained. The second method is to minimize the square sum of the F-norm of the error matrices produced by the algorithm and it is shown that the optimal step size exits uniquely and lies in an interval. Several numerical examples are given to illustrate the efficiency of the proposed approach.     

Minimal residual methods for large scale Lyapunov equations

Projection methods have emerged as competitive techniques for solving large scale matrix Lyapunov equations. We explore the numerical solution of this class of linear matrix equations when a Minimal Residual (MR) condition is used during the projection step. We derive both a new direct method, and a preconditioned operator-oriented iterative solver based on CGLS, for solving the projected reduced least squares problem. Numerical experiments with benchmark problems show the effectiveness of an MR approach over a Galerkin procedure using the same approximation space.     

A boundary only procedure for time-dependent diffusion equations

In the recent literature, the boundary element method (BEM) is extensively used to solve time-dependent partial differential equations. However, most of these formulations yield algorithms where one has to include all interior points in the computation process if finite difference procedures are used to approximate the temporal derivative. This obviously restricts the advantages of the BEM, which is mainly considered to be a boundary only algorithm for time-independent problems. A new algorithm is demonstrated here, which extends the boundary only nature of the method to time-dependent partial differential equations. Using this procedure, one can reduce the finite difference time integration algorithm, generated in a standard manner, to a boundary only process. The proposed method is demonstrated with considerable success for diffusion problems. Results obtained in these applications are presented comparatively with analytical and other boundary element time integration procedures. The algorithm proposed may utilize several coordinate functions in the secondary reduction phase of the formulation. A summary of such functions is described here and performances of these functions are tested and compared in three applications. It is shown that some coordinate functions perform better than others under certain conditions. Using these results, we propose a general coordinate function, which may be used with satisfactory results in all parabolic partial differential equation applications.     

Solutions of symmetry‐constrained least‐squares problems

In this paper, two new matrix‐form iterative methods are presented to solve the least‐squares problem: and matrix nearness problem: where matrices and are given; ??₁ and ??₂ are the set of constraint matrices, such as symmetric, skew symmetric, bisymmetric and centrosymmetric matrices sets and S_XY is the solution pair set of the minimum residual problem. These new matrix‐form iterative methods have also faster convergence rate and higher accuracy than the matrix‐form iterative methods proposed by Peng and Peng (Numer. Linear Algebra Appl. 2006; 13 : 473–485) for solving the linear matrix equation AXB+CYD=E. Paige's algorithms, which are based on the bidiagonalization procedure of Golub and Kahan, are used as the framework for deriving these new matrix‐form iterative methods. Some numerical examples illustrate the efficiency of the new matrix‐form iterative methods. Copyright © 2008 John Wiley & Sons, Ltd.     

AN ITERATIVE BEM FOR THE INVERSE PROBLEM OF DETECTING CORROSION IN A PIPE

In this paper, we consider an inverse problem of determining the corrosion occurring in an inaccessible interior part of a pipe from the measurements on the outer boundary. The problem is modelled by Laplace's equation with an unknowm term γ in the boundary condition on the inner boundary. Based on the Maz'ya iterative algorithm, a regularized BEM method is proposed for obtaining approximate solutions for this inverse problem. The numerical results show that our method can be easily realized and is quite effective.     

A moving pseudo‐boundary method of fundamental solutions for void detection

We propose a new moving pseudo‐boundary method of fundamental solutions (MFS) for the determination of the boundary of a void. This problem can be modeled as an inverse boundary value problem for harmonic functions. The algorithm for imaging the interior of the medium also makes use of radial polar parametrization of the unknown void shape in two dimensions. The center of this radial polar parametrization is considered to be unknown. We also include the contraction and dilation factors to be part of the unknowns in the resulting nonlinear least‐squares problem. This approach addresses the major problem of locating the pseudo‐boundary in the MFS in a natural way, because the inverse problem in question is nonlinear anyway. The feasibility of this new method is illustrated by several numerical examples. © 2012 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq, 2013     

Improved Gram–Schmidt Type Downdating Methods

The problem of deleting a row from a Q–R factorization (called downdating) using Gram–Schmidt orthogonalization is intimately connected to using classical iterative methods to solve a least squares problem with the orthogonal factor as the coefficient matrix. Past approaches to downdating have focused upon accurate computation of the residual of that least squares problem, then finding a unit vector in the direction of the residual that becomes a new column for the orthogonal factor. It is also important to compute the solution vector of the related least squares problem accurately, as that vector must be used in the downdating process to maintain good backward error in the new factorization. Using this observation, new algorithms are proposed. One of the new algorithms proposed is a modification of one due to Yoo and Park [BIT, 36:161–181, 1996]. That algorithm is shown to be a Gram–Schmidt procedure. Also presented are new results that bound the loss of orthogonality after downdating. An error analysis shows that the proposed algorithms’ behavior in floating point arithmetic is close to their behavior in exact arithmetic. Experiments show that the changes proposed in this paper can have a dramatic impact upon the accuracy of the downdated Q–R decomposition. AMS subject classification (2000) 65F20, 65F25     

Using Efficient Anchoring Points for Generating Search Directions in Interior Multiobjective Linear Programming

This paper modifies the affine-scaling primal algorithm to multiobjective linear programming (MOLP) problems. The modification is based on generating search directions in the form of projected gradients augmented by search directions pointing toward what we refer to as anchoring points. These anchoring points are located on the boundary of the feasible region and, together with the current, interior, iterate, define a cone in which we make the next step towards a solution of the MOLP problem. These anchoring points can be generated in more than one way. In this paper we present an approach that generates efficient anchoring points where the choice of termination solution available to the decision maker at each iteration consists of a set of efficient solutions. This set of efficient solutions is being updated during the iterative process so that only the most preferred solutions are retained for future considerations. Current MOLP algorithms are simplex-based and make their progress toward the optimal solution by following an exterior trajectory along the vertices of the constraints polytope. Since the proposed algorithm makes its progress through the interior of the constraints polytope, there is no need for vertex information and, therefore, the search for an acceptable solution may prove less sensitive to problem size. We refer to the resulting class of MOLP algorithms that are based on the affine-scaling primal algorithm as affine-scaling interior multiobjective linear programming (ASIMOLP) algorithms.     

A local iteration scheme for nonlinear two-dimensional steady-state heat conduction: a BEM approach

An efficient algorithm is proposed to solve the steady-state nonlinear heat conduction equation using the boundary element method (BEM). Nonlinearity of the heat conduction equation arises from nonlinear boundary conditions and temperature dependence of thermal conductivity. Using Kirchhoff's transformation, the case of temperature dependence of thermal conductivity can be transformed to the nonlinear boundary conditions case. Applying the BEM technique, the resulting matrix equation becomes nonlinear. The nonlinearity, however, only involves the boundary nodes that have nonlinearboundary conditions. The proposed local iterative scheme reduces the entire BEM matrix equation to a smaller matrix equation whose rank is the same as the number of boundary nodes with nonlinear boundary conditions. The Newton-Raphson iteration scheme is used to solve the reduced nonlinear matrix equation. The local iterative scheme is first applied to two one-dimensional problems (analytical solutions are possible) with different nonlinear boundary conditions. It is then applied to a two-region problem. Finally, the local iterative scheme is applied to two cavity problems in which radiation plays a role in the heat transfer.     

一类Riccati矩阵方程广义自反解的双迭代算法

本文研究了一类Riccati矩阵方程广义自反解的数值计算问题.利用牛顿算法将Riccati矩阵方程的广义自反解问题转化为线性矩阵方程的广义自反解或者广义自反最小二乘解问题,再利用修正共轭梯度法计算后一问题,获得了求Riccati矩阵方程的广义自反解的双迭代算法.拓宽了求解非线性矩阵方程的迭代算法.数值算例表明双迭代算法是有效的.     

含高次逆幂的矩阵方程对称解的双迭代算法

本文研究了在控制理论和随机滤波等领域中遇到的一类含高次逆幂的矩阵方程的等价矩阵方程对称解的数值计算问题.采用牛顿算法求等价矩阵方程的对称解,并采用修正共轭梯度法求由牛顿算法每一步迭代计算导出的线性矩阵方程的对称解或者对称最小二乘解,建立了求这类矩阵方程对称解的双迭代算法,数值算例验证了双迭代算法是有效的.     

Two-stage least squares and indirect least squares algorithms for simultaneous equations models

This paper analyzes the solution of simultaneous equations models. Efficient algorithms for the two-stage least squares method using QR-decomposition are developed and studied. The reduction of the execution time when the structure of the matrices in each equation is exploited is analyzed theoretically and experimentally. An efficient algorithm for the indirect least squares method is developed. Some techniques are used to accelerate the solution of the problem: parallel versions for multicore systems, and extensive use of the MKL library, thus obtaining efficient, portable versions of the algorithms.     

A method for solving an exterior three-dimensional boundary value problem for the Laplace equation

We develop and experimentally study the algorithms for solving three-dimensionalmixed boundary value problems for the Laplace equation in unbounded domains. These algorithms are based on the combined use of the finite elementmethod and an integral representation of the solution in a homogeneous space. The proposed approach consists in the use of the Schwarz alternating method with consecutive solution of the interior and exterior boundary value problems in the intersecting subdomains on whose adjoining boundaries the iterated interface conditions are imposed. The convergence of the iterative method is proved. The convergence rate of the iterative process is studied analytically in the case when the subdomains are spherical layers with the known exact representations of all consecutive approximations. In this model case, the influence of the algorithm parameters on the method efficiency is analyzed. The approach under study is implemented for solving a problem with a sophisticated configuration of boundaries while using a high precision finite elementmethod to solve the interior boundary value problems. The convergence rate of the iterations and the achieved accuracy of the computations are illustrated with some numerical experiments.     

Solving the minimal least squares problem subject to bounds on the variables

A computational procedure is developed for determining the solution of minimal length to a linear least squares problem subject to bounds on the variables. In the first stage, a solution to the least squares problem is computed and then in the second stage, the solution of minimal length is determined. The objective function in each step is minimized by an active set method adapted to the special structure of the problem.The systems of linear equations satisfied by the descent direction and the Lagrange multipliers in the minimization algorithm are solved by direct methods based on QR decompositions or iterative preconditioned conjugate gradient methods. The direct and the iterative methods are compared in numerical experiments, where the solutions are sought to a sequence of related, minimal least squares problems subject to bounds on the variables. The application of the iterative methods to large, sparse problems is discussed briefly.This work was supported by The National Swedish Board for Technical Development under contract dnr 80-3341.     

CP decomposition for tensors via alternating least squares with QR decomposition

The CP tensor decomposition is used in applications such as machine learning and signal processing to discover latent low-rank structure in multidimensional data. Computing a CP decomposition via an alternating least squares (ALS) method reduces the problem to several linear least squares problems. The standard way to solve these linear least squares subproblems is to use the normal equations, which inherit special tensor structure that can be exploited for computational efficiency. However, the normal equations are sensitive to numerical ill-conditioning, which can compromise the results of the decomposition. In this paper, we develop versions of the CP-ALS algorithm using the QR decomposition and the singular value decomposition, which are more numerically stable than the normal equations, to solve the linear least squares problems. Our algorithms utilize the tensor structure of the CP-ALS subproblems efficiently, have the same complexity as the standard CP-ALS algorithm when the input is dense and the rank is small, and are shown via examples to produce more stable results when ill-conditioning is present. Our MATLAB implementation achieves the same running time as the standard algorithm for small ranks, and we show that the new methods can obtain lower approximation error.     

Finite iterative method for solving coupled Sylvester-transpose matrix equations

This paper is concerned with solutions to the so-called coupled Sylvester-transpose matrix equations, which include the generalized Sylvester matrix equation and Lyapunov matrix equation as special cases. By extending the idea of conjugate gradient method, an iterative algorithm is constructed to solve this kind of coupled matrix equations. When the considered matrix equations are consistent, for any initial matrix group, a solution group can be obtained within finite iteration steps in the absence of roundoff errors. The least Frobenius norm solution group of the coupled Sylvester-transpose matrix equations can be derived when a suitable initial matrix group is chosen. By applying the proposed algorithm, the optimal approximation solution group to a given matrix group can be obtained by finding the least Frobenius norm solution group of new general coupled matrix equations. Finally, a numerical example is given to illustrate that the algorithm is effective.