Spectral estimates for unreduced symmetric KKT systems arising from Interior Point methods

We consider symmetrized Karush–Kuhn–Tucker systems arising in the solution of convex quadratic programming problems in standard form by Interior Point methods. Their coefficient matrices usually have 3 × 3 block structure, and under suitable conditions on both the quadratic programming problem and the solution, they are nonsingular in the limit. We present new spectral estimates for these matrices: the new bounds are established for the unpreconditioned matrices and for the matrices preconditioned by symmetric positive definite augmented preconditioners. Some of the obtained results complete the analysis recently given by Greif, Moulding, and Orban in [SIAM J. Optim., 24 (2014), pp. 49‐83]. The sharpness of the new estimates is illustrated by numerical experiments. Copyright © 2016 John Wiley & Sons, Ltd.     

On the eigenvalues of a class of saddle point matrices

We study spectral properties of a class of block 2 × 2 matrices that arise in the solution of saddle point problems. These matrices are obtained by a sign change in the second block equation of the symmetric saddle point linear system. We give conditions for having a (positive) real spectrum and for ensuring diagonalizability of the matrix. In particular, we show that these properties hold for the discrete Stokes operator, and we discuss the implications of our characterization for augmented Lagrangian formulations, for Krylov subspace solvers and for certain types of preconditioners. The work of this author was supported in part by the National Science Foundation grant DMS-0207599 Revision dated 5 December 2005.     

Eigenvalue estimates for preconditioned saddle point matrices

New accurate eigenvalue bounds for symmetric matrices of saddle point form are derived and applied for both unpreconditioned and preconditioned versions of the matrices. The estimates enable a better understanding of how preconditioners should be chosen. The preconditioners provide efficient iterative solution of the corresponding linear systems with, for some important applications, an optimal order of computational complexity. The methods are applied for Stokes problem and for linear elasticity problems. Copyright © 2005 John Wiley & Sons, Ltd.     

A general approach to analyse preconditioners for two‐by‐two block matrices

Two‐by‐two block matrices arise in various applications, such as in domain decomposition methods or when solving boundary value problems discretised by finite elements from the separation of the node set of the mesh into ‘fine’ and ‘coarse’ nodes. Matrices with such a structure, in saddle point form arise also in mixed variable finite element methods and in constrained optimisation problems. A general algebraic approach to construct, analyse and control the accuracy of preconditioners for matrices in two‐by‐two block form is presented. This includes both symmetric and nonsymmetric matrices, as well as indefinite matrices. The action of the preconditioners can involve element‐by‐element approximations and/or geometric or algebraic multigrid/multilevel methods. Copyright © 2011 John Wiley & Sons, Ltd.     

On the eigenvalue distribution of preconditioned nonsymmetric saddle point matrices

In this paper, we derive bounds for the complex eigenvalues of a nonsymmetric saddle point matrix with a symmetric positive semidefinite (2,2) block, that extend the corresponding previous bounds obtained by Bergamaschi. For the nonsymmetric saddle point problem, we propose a block diagonal preconditioner for the conjugate gradient method in a nonstandard inner product. Numerical experiments are also included to test the performance of the presented preconditioner. Copyright © 2013 John Wiley & Sons, Ltd.     

Towards an optimal condition number of certain augmented Lagrangian‐type saddle‐point matrices

We present an analysis for minimizing the condition number of nonsingular parameter‐dependent 2 × 2 block‐structured saddle‐point matrices with a maximally rank‐deficient (1,1) block. The matrices arise from an augmented Lagrangian approach. Using quasidirect sums, we show that a decomposition akin to simultaneous diagonalization leads to an optimization based on the extremal nonzero eigenvalues and singular values of the associated block matrices. Bounds on the condition number of the parameter‐dependent matrix are obtained, and we demonstrate their tightness on some numerical examples. Copyright © 2016 John Wiley & Sons, Ltd.     

A null-space approach for large-scale symmetric saddle point systems with a small and non zero (2, 2) block

Numerical Algorithms - Null-space methods have long been used to solve large sparse n × n symmetric saddle point systems of equations in which the (2, 2) block is zero. This paper focuses on...     

Preordering saddle‐point systems for sparse LDLT factorization without pivoting

This paper focuses on efficiently solving large sparse symmetric indefinite systems of linear equations in saddle‐point form using a fill‐reducing ordering technique with a direct solver. Row and column permutations partition the saddle‐point matrix into a block structure constituting a priori pivots of order 1 and 2. The partitioned matrix is compressed by treating each nonzero block as a single entry, and a fill‐reducing ordering is applied to the corresponding compressed graph. It is shown that, provided the saddle‐point matrix satisfies certain criteria, a block LDL^T factorization can be computed using the resulting pivot sequence without modification. Numerical results for a range of problems from practical applications using a modern sparse direct solver are presented to illustrate the effectiveness of the approach.     

Conjugate gradient method for dual-dual mixed formulations

We deal with the iterative solution of linear systems arising from so-called dual-dual mixed finite element formulations. The linear systems are of a two-fold saddle point structure; they are indefinite and ill-conditioned. We define a special inner product that makes matrices of the two-fold saddle point structure, after a specific transformation, symmetric and positive definite. Therefore, the conjugate gradient method with this special inner product can be used as iterative solver. For a model problem, we propose a preconditioner which leads to a bounded number of CG-iterations. Numerical experiments for our model problem confirming the theoretical results are also reported.

     

Kronecker product approximation preconditioners for convection–diffusion model problems

We consider the iterative solution of linear systems arising from four convection–diffusion model problems: scalar convection–diffusion problem, Stokes problem, Oseen problem and Navier–Stokes problem. We design preconditioners for these model problems that are based on Kronecker product approximations (KPAs). For this we first identify explicit Kronecker product structure of the coefficient matrices, in particular for the convection term. For the latter three model cases, the coefficient matrices have a 2 × 2 block structure, where each block is a Kronecker product or a summation of several Kronecker products. We then use this structure to design a block diagonal preconditioner, a block triangular preconditioner and a constraint preconditioner. Numerical experiments show the efficiency of the three KPA preconditioners, and in particular of the constraint preconditioner that usually outperforms the other two. This can be explained by the relationship that exists between these three preconditioners: the constraint preconditioner can be regarded as a modification of the block triangular preconditioner, which at its turn is a modification of the block diagonal preconditioner based on the cell Reynolds number. Copyright © 2009 John Wiley & Sons, Ltd.     

Analysis of iterative methods for saddle point problems: a unified approach

In this paper two classes of iterative methods for saddle point problems are considered: inexact Uzawa algorithms and a class of methods with symmetric preconditioners. In both cases the iteration matrix can be transformed to a symmetric matrix by block diagonal matrices, a simple but essential observation which allows one to estimate the convergence rate of both classes by studying associated eigenvalue problems. The obtained estimates apply for a wider range of situations and are partially sharper than the known estimates in literature. A few numerical tests are given which confirm the sharpness of the estimates.

     

Block and joint ℋ︁-matrix preconditioners for the Oseen equations

For saddle point problems in fluid dynamics, many preconditioners in the literature exploit the block structure of the problem to construct block diagonal or block triangular preconditioners. The performance of such preconditioners depends on whether fast, approximate solvers for the linear systems on the block diagonal as well as for the Schur complement are available. We will construct these efficient preconditioners using hierarchical matrix techniques in which fully populated matrices are approximated by blockwise low rank approximations. We will compare such block preconditioners with those obtained through a completely different approach where the given block structure is not used but a domain-decomposition based ℋ︁-LU factorization is constructed for the complete system matrix. Preconditioners resulting from these two approaches will be discussed and compared through numerical results. (© 2006 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

On eigenvalue distribution of constraint‐preconditioned symmetric saddle point matrices

This paper is devoted to the analysis of the eigenvalue distribution of two classes of block preconditioners for the generalized saddle point problem. Most of the bounds developed improve those appeared in previously published works. Numerical results onto a realistic test problem give evidence of the effectiveness of the estimates on the spectrum of preconditioned matrices. Copyright © 2011 John Wiley & Sons, Ltd.     

A practical solution for KKT systems

We consider KKT systems of linear equations with a 2 × 2 block indefinite matrix whose (2, 2) block is zero. Such systems arise in many applications. Treating such matrices would encounter some intricacies, especially when its (1, 1) block, i.e., the stiffness matrix in term of computational mechanics, is rank-deficient. It is the rank-deficiency of the stiffness matrix that leads to the so-called rigid-displacement issue. This is believed to be one of the main reasons that many programmers would unwillingly give up the Lagrange multiplier method but select the penalty method. Based on the Sherman–Morrison formula and the conventional LDL^T decomposition for symmetric positive definite matrices, a robust direct solution is proposed, which is amenable to the conventional finite element codes, competent for both nonsingular and singular stiffness matrices, and particularly suitable to parallel computation. As a paradigm, the application to the element-free Galerkin method (EFGM) with the moving least squares interpolation is illustrated. Funded by the National Natural Science Foundation of China (NSFC), Project no. 90510019.     

Uzawa type algorithms for nonsymmetric saddle point problems

In this paper, we consider iterative algorithms of Uzawa type for solving linear nonsymmetric saddle point problems. Specifically, we consider systems, written as usual in block form, where the upper left block is an invertible linear operator with positive definite symmetric part. Such saddle point problems arise, for example, in certain finite element and finite difference discretizations of Navier-Stokes equations, Oseen equations, and mixed finite element discretization of second order convection-diffusion problems. We consider two algorithms, each of which utilizes a preconditioner for the operator in the upper left block. Convergence results for the algorithms are established in appropriate norms. The convergence of one of the algorithms is shown assuming only that the preconditioner is spectrally equivalent to the inverse of the symmetric part of the operator. The other algorithm is shown to converge provided that the preconditioner is a sufficiently accurate approximation of the inverse of the upper left block. Applications to the solution of steady-state Navier-Stokes equations are discussed, and, finally, the results of numerical experiments involving the algorithms are presented.

     

广义鞍点问题的块三角预条件子

本文对Golub和Yuan(2002)中给出的ST分解推广到广义鞍点问题上,给出了三种块预条件子,并重点分析了其中两种预条件子应用到广义鞍点问题上所得到的对称正定阵,得出了其一般的性质并重点研究了预处理矩阵条件数的上界,最后给出了数值算例.     

On nonsymmetric saddle point matrices that allow conjugate gradient iterations

Linear systems in saddle point form are usually highly indefinite,which often slows down iterative solvers such as Krylov subspace methods. It has been noted by several authors that negating the second block row of a symmetric indefinite saddle point matrix leads to a nonsymmetric matrix ${{\mathcal A}}Linear systems in saddle point form are usually highly indefinite,which often slows down iterative solvers such as Krylov subspace methods. It has been noted by several authors that negating the second block row of a symmetric indefinite saddle point matrix leads to a nonsymmetric matrix whose spectrum is entirely contained in the right half plane. In this paper we study conditions so that is diagonalizable with a real and positive spectrum. These conditions are based on necessary and sufficient conditions for positive definiteness of a certain bilinear form,with respect to which is symmetric. In case the latter conditions are satisfied, there exists a well defined conjugate gradient (CG) method for solving linear systems with . We give an efficient implementation of this method, discuss practical issues such as error bounds, and present numerical experiments. In memory of Gene Golub (1932–2007), our wonderful friend and colleague, who had a great interest in the conjugate gradient method and the numerical solution of saddle point problems. The work of J?rg Liesen was supported by the Emmy Noether-Program and the Heisenberg-Program of the Deutsche Forschungsgemeinschaft.     

Symmetric-triangular decomposition and its applications part II: Preconditioners for indefinite systems

As an application of the symmetric-triangular (ST) decomposition given by Golub and Yuan (2001) and Strang (2003), three block ST preconditioners are discussed here for saddle point problems. All three preconditioners transform saddle point problems into a symmetric and positive definite system. The condition number of the three symmetric and positive definite systems are estimated. Therefore, numerical methods for symmetric and positive definite systems can be applied to solve saddle point problems indirectly. A numerical example for the symmetric indefinite system from the finite element approximation to the Stokes equation is given. Finally, some comments are given as well. AMS subject classification (2000) 65F10     

Preconditioning discretizations of systems of partial differential equations

This survey paper is based on three talks given by the second author at the London Mathematical Society Durham Symposium on Computational Linear Algebra for Partial Differential Equations in the summer of 2008. The main focus will be on an abstract approach to the construction of preconditioners for symmetric linear systems in a Hilbert space setting. Typical examples that are covered by this theory are systems of partial differential equations which correspond to saddle point problems. We will argue that the mapping properties of the coefficient operators suggest that block diagonal preconditioners are natural choices for these systems. To illustrate our approach a number of examples will be considered. In particular, parameter‐dependent systems arising in areas like incompressible flow, linear elasticity, and optimal control theory will be studied. The paper contains analysis of several models which have previously been discussed in the literature. However, here each example is discussed with reference to a more unified abstract approach. Copyright © 2010 John Wiley & Sons, Ltd.     

Lorentzian geometry for 2 × 2 real matrices

We give an interpretation and a natural proof of the lemma of Herglotz on 2 × 2 symmetric matrices from the point of view of 3-dimensional Lorentz geometry and 2-dimensional hyperbolic geometry. This interpretation leads naturally to an analogous result on 2 × 2 matrices with trace 0.