On generalized successive overrelaxation methods for augmented linear systems

For the augmented system of linear equations, Golub, Wu and Yuan recently studied an SOR-like method (BIT 41(2001)71–85). By further accelerating it with another parameter, in this paper we present a generalized SOR (GSOR) method for the augmented linear system. We prove its convergence under suitable restrictions on the iteration parameters, and determine its optimal iteration parameters and the corresponding optimal convergence factor. Theoretical analyses show that the GSOR method has faster asymptotic convergence rate than the SOR-like method. Also numerical results show that the GSOR method is more effective than the SOR-like method when they are applied to solve the augmented linear system. This GSOR method is further generalized to obtain a framework of the relaxed splitting iterative methods for solving both symmetric and nonsymmetric augmented linear systems by using the techniques of vector extrapolation, matrix relaxation and inexact iteration. Besides, we also demonstrate a complete version about the convergence theory of the SOR-like method. Subsidized by The Special Funds For Major State Basic Research Projects (No. G1999032803) and The National Natural Science Foundation (No. 10471146), P.R. China     

Convergence of a generalized MSSOR method for augmented systems

Recently, Wu et al. [S.-L. Wu, T.-Z. Huang, X.-L. Zhao, A modified SSOR iterative method for augmented systems, J. Comput. Appl. Math. 228 (1) (2009) 424-433] introduced a modified SSOR (MSSOR) method for augmented systems. In this paper, we establish a generalized MSSOR (GMSSOR) method for solving the large sparse augmented systems of linear equations, which is the extension of the MSSOR method. Furthermore, the convergence of the GMSSOR method for augmented systems is analyzed and numerical experiments are carried out, which show that the GMSSOR method with appropriate parameters has a faster convergence rate than the MSSOR method with optimal parameters.     

SOR-like Methods for Augmented Systems

Several SOR-like methods are proposed for solving augmented systems. These have many different applications in scientific computing, for example, constrained optimization and the finite element method for solving the Stokes equation. The convergence and the choice of optimal parameter for these algorithms are studied. The convergence and divergence regions for some algorithms are given, and the new algorithms are applied to solve the Stokes equations as well.     

A note on an SOR-like method for augmented systems

Golub et al. (2001, BIT, 41, 71–85) gave a generalizedsuccessive over-relaxation method for the augmented systems.In this paper, the connection between the SOR-like method andthe preconditioned conjugate gradient (PCG) method for the augmentedsystems is investigated. It is shown that the PCG method isat least as accurate (fast) as the SOR-like method. Numericalexamples demonstrate that the PCG method is much faster thanthe SOR-like method.     

A generalized modified SOR-like method for the singular saddle point problems

Recently, Guo et al. proposed a modified SOR-like (MSOR-like) iteration method for solving the nonsingular saddle point problem. In this paper, we further prove the semi-convergence of this method when it is applied to solve the singular saddle point problems under suitable conditions on the involved iteration parameters. Moreover, the optimal iteration parameters and the corresponding optimal semi-convergence factor for the MSOR-like method are determined. In addition, numerical experiments are used to show the feasibility and effectiveness of the MSOR-like method for solving singular saddle point problems, arising from the incompressible flow problems.     

A special Hermitian and skew-Hermitian splitting method for image restoration

In this paper, we consider an ill-posed image restoration problem with a noise contaminated observation, and a known convolution kernel. A special Hermitian and skew-Hermitian splitting (HSS) iterative method is established for solving the linear systems from image restoration. Our approach is based on an augmented system formulation. The convergence and operation cost of the special HSS iterative method for image restoration problems are discussed. The optimal parameter minimizing the spectral radius of the iteration matrix is derived. We present a detailed algorithm for image restoration problems. Numerical examples are given to demonstrate the performance of the presented method. Finally, the SOR acceleration scheme for the special HSS iterative method is discussed.     

非Hermitian正定线性方程组的外推的HSS迭代方法

为了高效地求解大型稀疏非Hermitian正定线性方程组，在白中治、Golub和Ng提出的Hermitian和反Hermitian分裂（HSS）迭代法的基础上，通过引入新的参数并结合迭代法的松弛技术，对HSS迭代方法进行加速，提出了一种新的外推的HSS迭代方法（EHSS），并研究了该方法的收敛性.数值例子表明：通过参数值的选择，新方法比HSS方法具有更快的收敛速度和更少的迭代次数，选择了合适的参数值后，可以提高HSS方法的收敛效率.     

The selection of the optimal parameter in the modulus-based matrix splitting algorithm for linear complementarity problems

The modulus-based matrix splitting (MMS) algorithm is effective to solve linear complementarity problems (Bai in Numer Linear Algebra Appl 17: 917–933, 2010). This algorithm is parameter dependent, and previous studies mainly focus on giving the convergence interval of the iteration parameter. Yet the specific selection approach of the optimal parameter has not been systematically studied due to the nonlinearity of the algorithm. In this work, we first propose a novel and simple strategy for obtaining the optimal parameter of the MMS algorithm by merely solving two quadratic equations in each iteration. Further, we figure out the interval of optimal parameter which is iteration independent and give a practical choice of optimal parameter to avoid iteration-based computations. Compared with the experimental optimal parameter, the numerical results from three problems, including the Signorini problem of the Laplacian, show the feasibility, effectiveness and efficiency of the proposed strategy.

     

GSOR方法最优迭代参数的简化推导

对于增广线性系统,Bai等研究了广义SOR方法（Bai Z Z,Parlett B,Wang Z Q.On generaliged successive overrelaxation methods for augmented linear systems.Numerische Mathematik,2005,102（1）：1-38）,并得到其最优迭代参数.给出了另外一种推导最优迭代参数的简化方法,这种方法对于求解其他参数加速定常迭代方法的最优迭代参数非常有意义.     

HSS METHOD WITH A COMPLEX PARAMETER FOR THE SOLUTION OF COMPLEX LINEAR SYSTEM

In this paper, a complex parameter is employed in the Hermitian and skew-Hermitian splitting (HSS) method (Bai, Golub and Ng: SIAM J. Matrix Anal. Appl., 24(2003), 603-626) for solving the complex linear system $Ax=f$. The convergence of the resulting method is proved when the spectrum of the matrix $A$ lie in the right upper (or lower) part of the complex plane. We also derive an upper bound of the spectral radius of the HSS iteration matrix, and an estimated optimal parameter $α$(denoted by $α_{est}$) of this upper bound is presented. Numerical experiments on two modified model problems show that the HSS method with $α_{est}$has a smaller spectral radius than that with the real parameter which minimizes the corresponding upper bound. In particular, for the 'dominant' imaginary part of the matrix $A$, this improvement is considerable. We also test the GMRES method preconditioned by the HSS preconditioning matrix with our parameter $α_{est}$.     

A Mixed Logarithmic Barrier-Augmented Lagrangian Method for Nonlinear Optimization

We present a primal–dual algorithm for solving a constrained optimization problem. This method is based on a Newtonian method applied to a sequence of perturbed KKT systems. These systems follow from a reformulation of the initial problem under the form of a sequence of penalized problems, by introducing an augmented Lagrangian for handling the equality constraints and a log-barrier penalty for the inequalities. We detail the updating rules for monitoring the different parameters (Lagrange multiplier estimate, quadratic penalty and log-barrier parameter), in order to get strong global convergence properties. We show that one advantage of this approach is that it introduces a natural regularization of the linear system to solve at each iteration, for the solution of a problem with a rank deficient Jacobian of constraints. The numerical experiments show the good practical performances of the proposed method especially for degenerate problems.     

Generalized shift-splitting iteration method for a class of two-by-two linear systems

In this paper, we propose a generalized-shift splitting (GSS) iteration method for solving a broad class of two-by-two linear systems. The convergence theory of the GSS iteration method is established and the spectral properties of the corresponding preconditioned matrix of some special choices for the parameter matrices are analyzed. In addition, the optimal choice of the iterative parameter is discussed. Numerical experiments are used to verify the validity of the theoretical results and the effectiveness of the new method.     

The cyclic reduction algorithm: from Poisson equation to stochastic processes and beyond

Cyclic reduction is an algorithm invented by G. H. Golub and R. W. Hockney in the mid 1960s for solving linear systems related to the finite differences discretization of the Poisson equation over a rectangle. Among the algorithms of Gene Golub, it is one of the most versatile and powerful ever created. Recently, it has been applied to solve different problems from different applicative areas. In this paper we survey the main features of cyclic reduction, relate it to properties of analytic functions, recall its extension to solving more general finite and infinite linear systems, and different kinds of nonlinear matrix equations, including algebraic Riccati equations, with applications to Markov chains, queueing models and transport theory. Some new results concerning the convergence properties of cyclic reduction and its applicability are proved under very weak assumptions. New formulae for overcoming breakdown are provided.     

A single-step HSS method for non-Hermitian positive definite linear systems

In this paper, based on the Hermitian and skew-Hermitian splitting (HSS) iteration method, a single-step HSS (SHSS) iteration method is introduced to solve the non-Hermitian positive definite linear systems. Theoretical analysis shows that, under a loose restriction on the iteration parameter, the SHSS method is convergent to the unique solution of the linear system. Furthermore, we derive an upper bound for the spectral radius of the SHSS iteration matrix, and the quasi-optimal parameter is obtained to minimize the above upper bound. Numerical experiments are reported to the efficiency of the SHSS method; numerical comparisons show that the proposed SHSS method is superior to the HSS method under certain conditions.     

求解一类分块二阶线性方程组的QHSS迭代方法

本文将QHSS迭代方法运用于求解一类分块二阶线性方程组.通过适当地放宽QHSS迭代方法的收敛性条件,我们给出了用QHSS迭代方法求解一类分块二阶线性方程组的具体迭代格式,并证明了当系数矩阵中的(1,1)块对称半正定时该QHSS迭代方法的收敛性.我们还用数值实验验证了QHSS迭代方法的可行性和有效性.     

A class of Uzawa-SOR methods for saddle point problems

In this paper, we consider a class of Uzawa-SOR methods for saddle point problems, and prove the convergence of the proposed methods. We solve a lower triangular system per iteration in the proposed methods, instead of solving a linear equation Az=b. Actually, the new methods can be considered as an inexact iteration method with the Uzawa as the outer iteration and the SOR as the inner iteration. Although the proposed methods cannot achieve the same convergence rate as the GSOR methods proposed by Bai et al. [Z.-Z. Bai, B.N. Parlett, Z.-Q. Wang, On generalized successive overrelaxation methods for augmented linear systems, Numer. Math. 102 (2005) 1-38], but our proposed methods have less workloads per iteration step. Experimental results show that our proposed methods are feasible and effective.     

On generalized symmetric SOR method for augmented systems

In this paper, we establish the generalized symmetric SOR method (GSSOR) for solving the large sparse augmented systems of linear equations, which is the extension of the SSOR iteration method. The convergence of the GSSOR method for augmented systems is studied. Numerical resume shows that this method is effective.     

Diagonal and Toeplitz splitting iteration methods for diagonal‐plus‐Toeplitz linear systems from spatial fractional diffusion equations

The finite difference discretization of the spatial fractional diffusion equations gives discretized linear systems whose coefficient matrices have a diagonal‐plus‐Toeplitz structure. For solving these diagonal‐plus‐Toeplitz linear systems, we construct a class of diagonal and Toeplitz splitting iteration methods and establish its unconditional convergence theory. In particular, we derive a sharp upper bound about its asymptotic convergence rate and deduct the optimal value of its iteration parameter. The diagonal and Toeplitz splitting iteration method naturally leads to a diagonal and circulant splitting preconditioner. Analysis shows that the eigenvalues of the corresponding preconditioned matrix are clustered around 1, especially when the discretization step‐size h is small. Numerical results exhibit that the diagonal and circulant splitting preconditioner can significantly improve the convergence properties of GMRES and BiCGSTAB, and these preconditioned Krylov subspace iteration methods outperform the conjugate gradient method preconditioned by the approximate inverse circulant‐plus‐diagonal preconditioner proposed recently by Ng and Pan (M.K. Ng and J.‐Y. Pan, SIAM J. Sci. Comput. 2010;32:1442‐1464). Moreover, unlike this preconditioned conjugate gradient method, the preconditioned GMRES and BiCGSTAB methods show h‐independent convergence behavior even for the spatial fractional diffusion equations of discontinuous or big‐jump coefficients.     

A generalization of parameterized inexact Uzawa method for generalized saddle point problems

Recently, a class of parameterized inexact Uzawa methods has been proposed for generalized saddle point problems by Bai and Wang [Z.-Z. Bai, Z.-Q. Wang, On parameterized inexact Uzawa methods for generalized saddle point problems, Linear Algebra Appl. 428 (2008) 2900–2932], and a generalization of the inexact parameterized Uzawa method has been studied for augmented linear systems by Chen and Jiang [F. Chen, Y.-L. Jiang, A generalization of the inexact parameterized Uzawa methods for saddle point problems, Appl. Math. Comput. (2008)]. This paper is concerned about a generalization of the parameterized inexact Uzawa method for solving the generalized saddle point problems with nonzero (2, 2) blocks. Some new iterative methods are presented and their convergence are studied in depth. By choosing different parameter matrices, we derive a series of existing and new iterative methods, including the preconditioned Uzawa method, the inexact Uzawa method, the SOR-like method, the GSOR method, the GIAOR method, the PIU method, the APIU method and so on. Numerical experiments are used to demonstrate the feasibility and effectiveness of the generalized parameterized inexact Uzawa methods.     

A modified ASOR-like method for augmented linear systems

Numerical Algorithms - For solving an augmented linear system, Njeru and Guo presented an accelerated SOR-like (ASOR) method in (P. N. Njeru and X.-P. Guo. Accelerated SOR-like method for augmented...