Further study of alternating iterations for rectangular matrices

Theory of matrix splittings is a useful tool in the analysis of iterative methods for solving systems of linear equations. When two splittings are given, it is of interest to compare the spectral radii of the corresponding iteration matrices. This helps to arrive at the conclusion that which splitting should one choose so that one can reach the desired solution of accuracy or the exact solution in a faster way. In the case of many splittings are provided, the comparison of the spectral radii is time-consuming. Such a situation can be overcome by introducing another iteration scheme which converges to the same solution of interest in a much faster way. In this direction, the theory of alternating iterations for real rectangular matrices is recently proposed. In this note, some more results to the theory of alternating iterations are added. A comparison result of two different alternating iteration schemes is then presented which will help us to choose the iteration scheme that will guarantee the faster convergence of the alternating iteration scheme. In addition to these results, a comparison result for proper weak regular splittings is also obtained.     

Convergent nonnegative matrices and iterative methods for consistent linear systems

Summary In this paper we study linear stationary iterative methods with nonnegative iteration matrices for solving singular and consistent systems of linear equationsAx=b. The iteration matrices for the schemes are obtained via regular and weak regular splittings of the coefficients matrixA. In certain cases when only some necessary, but not sufficient, conditions for the convergence of the iterations schemes exist, we consider a transformation on the iteration matrices and obtain new iterative schemes which ensure convergence to a solution toAx=b. This transformation is parameter-dependent, and in the case where all the eigenvalues of the iteration matrix are real, we show how to choose this parameter so that the asymptotic convergence rate of the new schemes is optimal. Finally, some applications to the problem of computing the stationary distribution vector for a finite homogeneous ergodic Markov chain are discussed.Research sponsored in part by US Army Research Office     

A note on properties of splittings of singular symmetric positive semidefinite matrices

Summary. Recently, Benzi and Szyld have published an important paper [1] concerning the existence and uniqueness of splittings for singular matrices. However, the assertion in Theorem 3.9 on the inheriting property of P-regular splitting for singular symmetric positive semidefinite matrices seems to be incorrect. As a complement of paper [1], in this short note we point out that if a matrix T is resulted from a P-regular splitting of a symmetric positive semidefinite matrix A, then splittings induced by T are not all P-regular. Received January 7, 1999 / Published online December 19, 2000     

On M–multisplittings of singular M–matrices with application to Markov chains

Given a singular M–matrix of a linear system, convergent conditions under which iterative schemes based on M–multisplittings are studied. Two of those conditions, the index of the iteration matrix and its spectral radius are investigated and related to those of the M-matrix. Furthermore, a parallel multisplitting iteration scheme for solving singular linear systems is suggested which can be applied to practical problems such as Poisson and elasticity problems under certain boundary conditions, the Neumann problem, and in Markov chains. A discussion of that multisplitting scheme, based on Gauss–Seidel type splittings is given for computing the stationary distribution vector of Markov chains. In this case a computational viable algorithm can be constructed, since only the nonsingularity of one weighting matrix of the multisplitting is needed. © 1998 John Wiley & Sons, Ltd.     

Optimal parameters in the HSS‐like methods for saddle‐point problems

For the Hermitian and skew‐Hermitian splitting iteration method and its accelerated variant for solving the large sparse saddle‐point problems, we compute their quasi‐optimal iteration parameters and the corresponding quasi‐optimal convergence factors for the more practical but more difficult case that the (1, 1)‐block of the saddle‐point matrix is not algebraically equivalent to the identity matrix. In addition, the algebraic behaviors and the clustering properties of the eigenvalues of the preconditioned matrices with respect to these two iterations are investigated in detail, and the formulas for computing good iteration parameters are given under certain principle for optimizing the distribution of the eigenvalues. Copyright © 2008 John Wiley & Sons, Ltd.     

Modulus‐based matrix splitting iteration methods for linear complementarity problems

For the large sparse linear complementarity problems, by reformulating them as implicit fixed‐point equations based on splittings of the system matrices, we establish a class of modulus‐based matrix splitting iteration methods and prove their convergence when the system matrices are positive‐definite matrices and H₊‐matrices. These results naturally present convergence conditions for the symmetric positive‐definite matrices and the M‐matrices. Numerical results show that the modulus‐based relaxation methods are superior to the projected relaxation methods as well as the modified modulus method in computing efficiency. Copyright © 2009 John Wiley & Sons, Ltd.     

A class of block alternating splitting implicit iteration methods for double saddle point linear systems

In the framework of a special block alternating splitting implicit (BASI) iteration scheme for generalized saddle point problems, we establish some new iteration methods for solving double saddle point problems by means of a suitable partitioning strategy. Convergence analysis of the corresponding BASI iteration methods indicates that they are convergent unconditionally under certain weak requirements for the related matrix splittings, which are satisfied directly for our specific application to double saddle point problems. Numerical examples for liquid crystal director and time-harmonic eddy current models are presented to demonstrate the efficiency of the proposed BASI preconditioners to accelerate the GMRES method.     

H-Splittings and two-stage iterative methods

Summary Convergence of two-stage iterative methods for the solution of linear systems is studied. Convergence of the non-stationary method is shown if the number of inner iterations becomes sufficiently large. TheR ₁-factor of the two-stage method is related to the spectral radius of the iteration matrix of the outer splitting. Convergence is further studied for splittings ofH-matrices. These matrices are not necessarily monotone. Conditions on the splittings are given so that the two-stage method is convergent for any number of inner iterations.This work was supported in part by a Temple University Summer Research Fellowship.     

Respectively scaled HSS iteration methods for solving discretized spatial fractional diffusion equations

For the discrete linear systems resulted from the discretization of the one‐dimensional anisotropic spatial fractional diffusion equations of variable coefficients with the shifted finite‐difference formulas of the Grünwald–Letnikov type, we propose a class of respectively scaled Hermitian and skew‐Hermitian splitting iteration method and establish its asymptotic convergence theory. The corresponding induced matrix splitting preconditioner, through further replacements of the involved Toeplitz matrices with certain circulant matrices, leads to an economic variant that can be executed by fast Fourier transforms. Both theoretical analysis and numerical implementations show that this fast respectively scaled Hermitian and skew‐Hermitian splitting preconditioner can significantly improve the computational efficiency of the Krylov subspace iteration methods employed as effective linear solvers for the target discrete linear systems.     

On multisplitting methods for band matrices

Summary. We present new theoretical results on two classes of multisplitting methods for solving linear systems iteratively. These classes are based on overlapping blocks of the underlying coefficient matrix which is assumed to be a band matrix. We show that under suitable conditions the spectral radius of the iteration matrix does not depend on the weights of the method even if these weights are allowed to be negative. For a certain class of splittings we prove an optimality result for with respect to the weights provided that is an M–matrix. This result is based on the fact that the multisplitting method can be represented by a single splitting which in our situation surprisingly turns out to be a regular splitting. Furthermore we show by numerical examples that weighting factors may considerably improve the convergence. Received July 18, 1994 / Revised version received November 20, 1995     

Algebraic multigrid methods for Laplacians of graphs

Classical algebraic multigrid theory relies on the fact that the system matrix is positive definite. We extend this theory to cover the positive semidefinite case as well, by formulating semiconvergence results for these singular systems. For the class of irreducible diagonal dominant singular M-matrices we show that the requirements of the developed theory hold and that the coarse level systems are still of the same class, if the C/F-splitting is good enough. An important example for matrices that are irreducible diagonal dominant M-matrices are Laplacians of graphs. Recent shape optimizing methods for graph partitioning require to solve singular linear systems involving these Laplacians. We present convergence results as well as experimental results for numerous graphs arising from finite element discretizations with up to 10⁶ vertices.     

Shifted skew-symmetric iteration methods for nonsymmetric linear complementarity problems

We present a shifted skew-symmetric iteration method for solving the nonsymmetric positive definite or positive semidefinite linear complementarity problems. This method is based on the symmetric and skew-symmetric splitting of the system matrix, which has been adopted to establish efficient splitting iteration methods for solving the nonsymmetric systems of linear equations. Global convergence of the method is proved, and the corresponding inexact splitting iteration scheme is established and analyzed in detail. Numerical results show that the new methods are feasible and effective for solving large sparse and nonsymmetric linear complementarity problems.     

Block Two-stage Methods for Singular Systems and Markov Chains

The use of block two-stage methods for the iterative solution of consistent singular linear systems is studied. In these methods, suitable for parallel computations, different blocks, i.e., smaller linear systems, can be solved concurrently by different processors. Each of these smaller systems are solved by an (inner) iterative method. Hypotheses are provided for the convergence of non-stationary methods, i.e., when the number of inner iterations may vary from block to block and from one outer iteration to another. It is shown that the iteration matrix corresponding to one step of the block method is convergent, i.e., that its powers converge to a limit matrix. A theorem on the convergence of the infinite product of matrices with the same eigenspace corresponding to the eigenvalue 1 is proved, and later used as a tool in the convergence analysis of the block method. The methods studied can be used to solve any consistent singular system, including discretizations of certain differential equations. They can also be used to find stationary probability distribution of Markov chains. This last application is considered in detail.     

On convergence of two-stage splitting methods for linear complementarity problems

In this paper, we study the splitting method and two-stage splitting method for the linear complementarity problems. Convergence results for these two methods are presented when the system matrix is an H-matrix and the splittings used are H-splitting. Numerical experiments show that the two-stage splitting method has the same or even better numerical performance than the splitting method in some aspects under certain conditions.     

On regularized Hermitian splitting iteration methods for solving discretized almost‐isotropic spatial fractional diffusion equations

The shifted finite‐difference discretization of the one‐dimensional almost‐isotropic spatial fractional diffusion equation results in a discrete linear system whose coefficient matrix is a sum of two diagonal‐times‐Toeplitz matrices. For this kind of linear systems, we propose a class of regularized Hermitian splitting iteration methods and prove its asymptotic convergence under mild conditions. For appropriate circulant‐based approximation to the corresponding regularized Hermitian splitting preconditioner, we demonstrate that the induced fast regularized Hermitian splitting preconditioner possesses a favorable preconditioning property. Numerical results show that, when used to precondition Krylov subspace iteration methods such as generalized minimal residual and biconjugate gradient stabilized methods, the fast preconditioner significantly outperforms several existing ones.     

Coloring update methods

For linear update methods (such as SOR), a coloring method is introduced for which the multicolor iteration matrix has the same spectrum as the original iteration matrix. It applies to general linear systems, not necessarily arising from PDEs. When the iteration matrices are nonsingular, it is shown that they are similar to each other.     

Complex-extrapolated MHSS iteration method for singular complex symmetric linear systems

In this paper, by extrapolating the modified Hermitian and skew-Hermitian splitting (MHSS) iteration method with a complex relaxation parameter, a complex-extrapolated MHSS (CMHSS) iteration method is present for solving a class of complex singular symmetric of linear equations. We study the semi-convergence properties of the CMHSS iteration method and the extent of the optimal iterative parameters. Furthermore, the convergence conditions also hold for solving nonsingular complex systems. Numerical experiments are given to verify the effectiveness of the CMHSS iteration method for solving both singular and nonsingular complex symmetric systems.     

A solution to an inverse height and level characteristics problem

Given two sequences α and β of positive integers, it is shown that there exists a matrix with level characteristic α and height characteristic β if and only if β strictly majorizes α when reordered in non-increasing order. This generalizes a recently proven result for the case of matrices with simple singular vertices.     

Weighted pseudoinverses and weighted normal pseudosolutions with singular weights

Weighted pseudoinverses with singular weights can be defined by a system of matrix equations. For one of such definitions, necessary and sufficient conditions are given for the corresponding system to have a unique solution. Representations of the pseudoinverses in terms of the characteristic polynomials of symmetrizable and symmetric matrices, as well as their expansions in matrix power series or power products, are obtained. A relationship is found between the weighted pseudoinverses and the weighted normal pseudosolutions, and iterative methods for calculating both pseudoinverses and pseudosolutions are constructed. The properties of the weighted pseudoinverses with singular weights are shown to extend the corresponding properties of weighted pseudoinverses with positive definite weights.     

Product-type skew-Hermitian triangular splitting iteration methods for strongly non-Hermitian positive definite linear systems

By further generalizing the modified skew-Hermitian triangular splitting iteration methods studied in [L. Wang, Z.-Z. Bai, Skew-Hermitian triangular splitting iteration methods for non-Hermitian positive definite linear systems of strong skew-Hermitian parts, BIT Numer. Math. 44 (2004) 363-386], in this paper, we present a new iteration scheme, called the product-type skew-Hermitian triangular splitting iteration method, for solving the strongly non-Hermitian systems of linear equations with positive definite coefficient matrices. We discuss the convergence property and the optimal parameters of this method. Moreover, when it is applied to precondition the Krylov subspace methods, the preconditioning property of the product-type skew-Hermitian triangular splitting iteration is analyzed in detail. Numerical results show that the product-type skew-Hermitian triangular splitting iteration method can produce high-quality preconditioners for the Krylov subspace methods for solving large sparse positive definite systems of linear equations of strong skew-Hermitian parts.