Performance of ILU factorization preconditioners based on multisplittings

Summary. In this paper, we study the convergence of multisplitting methods associated with a multisplitting which is obtained from the ILU factorizations of a general H-matrix, and then we propose parallelizable ILU factorization preconditioners based on multisplittings for a block-tridiagonal H-matrix. We also describe a parallelization of preconditioned Krylov subspace methods with the ILU preconditioners based on multisplittings on distributed memory computers such as the Cray T3E. Lastly, parallel performance results of the preconditioned BiCGSTAB are provided to evaluate the efficiency of the ILU preconditioners based on multisplittings on the Cray T3E. Mathematics Subject Classification (2000):65F10, 65Y05, 65F50This work was supported by Korea Research Foundation Grant (KRF-2001-015-DP0051)     

Superlinearly convergent CG methods via equivalent preconditioning for nonsymmetric elliptic operators

Summary. The convergence of the conjugate gradient method is studied for preconditioned linear operator equations with nonsymmetric normal operators, with focus on elliptic convection-diffusion operators in Sobolev space. Superlinear convergence is proved first for equations whose preconditioned form is a compact perturbation of the identity in a Hilbert space. Then the same convergence result is verified for elliptic convection-diffusion equations using different preconditioning operators. The convergence factor involves the eigenvalues of the corresponding operators, for which an estimate is also given. The above results enable us to verify the mesh independence of the superlinear convergence estimates for suitable finite element discretizations of the convection-diffusion problems.Mathematics Subject Classification (2000): 65J10, 65F10, 65N15The second author was supported by the Hungarian Research Grant OTKA No. T. 043765.Dedicated to David M. Young on the occasion of his 80th birthday.     

On the preconditioning of matrices with skew-symmetric splittings

The rates of convergence of iterative methods with standard preconditioning techniques usually degrade when the skew-symmetric part S of the matrix is relatively large. In this paper, we address the issue of preconditioning matrices with such large skew-symmetric parts. The main idea of the preconditioner is to split the matrix into its symmetric and skew-symmetric parts and to invert the (shifted) skew-symmetric matrix. Successful use of the method requires the solution of a linear system with matrix I+S. An efficient method is developed using the normal equations, preconditioned by an incomplete orthogonal factorization.Numerical experiments on various systems arising in physics show that the reduction in terms of iteration count compensates for the additional work per iteration when compared to standard preconditioners.     

Circulant preconditioners for second order hyperbolic equations

Linear systems arising from implicit time discretizations and finite difference space discretizations of second-order hyperbolic equations in two dimensions are considered. We propose and analyze the use of circulant preconditioners for the solution of linear systems via preconditioned iterative methods such as the conjugate gradient method. Our motivation is to exploit the fast inversion of circulant systems with the Fast Fourier Transform (FFT). For second-order hyperbolic equations with initial and Dirichlet boundary conditions, we prove that the condition number of the preconditioned system is ofO() orO(m), where is the quotient between the time and space steps andm is the number of interior gridpoints in each direction. The results are extended to parabolic equations. Numerical experiments also indicate that the preconditioned systems exhibit favorable clustering of eigenvalues that leads to a fast convergence rate.     

Preconditioned Hermitian and skew-Hermitian splitting methods for non-Hermitian positive semidefinite linear systems

Summary. For the positive semidefinite system of linear equations of a block two-by-two structure, by making use of the Hermitian/skew-Hermitian splitting iteration technique we establish a class of preconditioned Hermitian/skew-Hermitian splitting iteration methods. Theoretical analysis shows that the new method converges unconditionally to the unique solution of the linear system. Moreover, the optimal choice of the involved iteration parameter and the corresponding asymptotic convergence rate are computed exactly. Numerical examples further confirm the correctness of the theory and the effectiveness of the method.Mathematics Subject Classification: 65F10, 65F50, CR: G1.3Subsidized by The Special Funds For Major State Basic Research Projects G1999032803Research supported, in part, by DOE-FC02-01ER4177Revised version received November 5, 2003     

An inexact primal–dual path following algorithm for convex quadratic SDP

We propose primal–dual path-following Mehrotra-type predictor–corrector methods for solving convex quadratic semidefinite programming (QSDP) problems of the form: , where is a self-adjoint positive semidefinite linear operator on , b∈R ^m , and is a linear map from to R ^m . At each interior-point iteration, the search direction is computed from a dense symmetric indefinite linear system (called the augmented equation) of dimension m + n(n + 1)/2. Such linear systems are typically very large and can only be solved by iterative methods. We propose three classes of preconditioners for the augmented equation, and show that the corresponding preconditioned matrices have favorable asymptotic eigenvalue distributions for fast convergence under suitable nondegeneracy assumptions. Numerical experiments on a variety of QSDPs with n up to 1600 are performed and the computational results show that our methods are efficient and robust. Research supported in part by Academic Research Grant R146-000-076-112.     

Combination preconditioning of saddle point systems for positive definiteness

Amongst recent contributions to preconditioning methods for saddle point systems, standard iterative methods in nonstandard inner products have been usefully employed. Krzy?anowski (Numerical Linear Algebra with Applications 2011; 18 :123–140) identified a two‐parameter family of preconditioners in this context and Stoll and Wathen (SIAM Journal on Matrix Analysis and Applications 2008; 30 :582–608) introduced combination preconditioning, where two preconditioners, self‐adjoint with respect to different inner products, can lead to further preconditioners and associated bilinear forms or inner products. Preconditioners that render the preconditioned saddle point matrix nonsymmetric but self‐adjoint with respect to a nonstandard inner product always allow a MINRES‐type method (‐PMINRES) to be applied in the relevant inner product. If the preconditioned matrix is also positive definite with respect to the inner product, a more efficient CG‐like method (‐PCG) can be reliably used. We establish eigenvalue expressions for Krzy?anowski preconditioners and show that for a specific choice of parameters, although the Krzy?anowski preconditioned saddle point matrix is self‐adjoint with respect to an inner product, it is never positive definite. We provide explicit expressions for the combination of certain preconditioners and prove the rather counterintuitive result that the combination of two specific preconditioners for which only ‐PMINRES can be reliably used leads to a preconditioner for which, for certain parameter choices, ‐PCG is reliably applicable. That is, combining two indefinite preconditioners can lead to a positive definite preconditioner. This combination preconditioner outperforms either of the two preconditioners from which it is formed for a number of test problems. Copyright © 2012 John Wiley & Sons, Ltd.     

Uniqueness of optimal controls inL ∞

We consider those functionsuL (a, b) which satisfyX=Ax+Bu, whereX satisfies certain prescribed interpolatory constraints. We show that there is au with smallest sup norm and that suchu are uniquely determined on a certain subset of [a, b]; more restrictions allow us to conclude thatX is uniquely determined on this set. An extension is given to certain linear control systems on the real line.Research supported in part by National Science Foundation Grant Nos. GP-43955 and MCS 75-05591.Research supported in part by National Science Foundation Grant Nos. GP-43955 and MPS 74-02292-A01.     

BCCB preconditioners for systems of BVM‐based numerical integrators

Boundary value methods (BVMs) for ordinary differential equations require the solution of non‐symmetric, large and sparse linear systems. In this paper, these systems are solved by using the generalized minimal residual (GMRES) method. A block‐circulant preconditioner with circulant blocks (BCCB preconditioner) is proposed to speed up the convergence rate of the GMRES method. The BCCB preconditioner is shown to be invertible when the BVM is A_k1,k2‐stable. The spectrum of the preconditioned matrix is clustered and therefore, the preconditioned GMRES method converges fast. Moreover, the operation cost in each iteration of the preconditioned GMRES method by using our BCCB preconditioner is less than that required by using block‐circulant preconditioners proposed earlier. In numerical experiments, we compare the number of iterations of various preconditioners. Copyright © 2003 John Wiley & Sons, Ltd.     

Global solutions of stochastic 2D Navier-Stokes equations with Lévy noise

In this paper, we prove the global existence and uniqueness of the strong and weak solutions for 2D Navier-Stokes equations on the torus perturbed by a Lévy process. The existence of invariant measure of the solutions are proved also. This work was supported by National Basic Research Program of China (Grant No. 2006CB8059000), Science Fund for Creative Research Groups (Grant No. 10721101), National Natural Science Foundation of China (Grant Nos. 10671197, 10671168), Science Foundation of Jiangsu Province (Grant Nos. BK2006032, 06-A-038, 07-333) and Key Lab of Random Complex Structures and Data Science, Chinese Academy of Sciences     

Preconditioning Indefinite Systems in Interior Point Methods for Optimization

Every Newton step in an interior-point method for optimization requires a solution of a symmetric indefinite system of linear equations. Most of today's codes apply direct solution methods to perform this task. The use of logarithmic barriers in interior point methods causes unavoidable ill-conditioning of linear systems and, hence, iterative methods fail to provide sufficient accuracy unless appropriately preconditioned. Two types of preconditioners which use some form of incomplete Cholesky factorization for indefinite systems are proposed in this paper. Although they involve significantly sparser factorizations than those used in direct approaches they still capture most of the numerical properties of the preconditioned system. The spectral analysis of the preconditioned matrix is performed: for convex optimization problems all the eigenvalues of this matrix are strictly positive. Numerical results are given for a set of public domain large linearly constrained convex quadratic programming problems with sizes reaching tens of thousands of variables. The analysis of these results reveals that the solution times for such problems on a modern PC are measured in minutes when direct methods are used and drop to seconds when iterative methods with appropriate preconditioners are used.     

A new approach for finding a basis for the splitting preconditioner for linear systems from interior point methods

The class of splitting preconditioners for the iterative solution of linear systems arising from Mehrotra’s predictor-corrector method for large scale linear programming problems needs to find a basis through a sophisticated process based on the application of a rectangular LU factorization. This class of splitting preconditioners works better near a solution of the linear programming problem when the matrices are highly ill-conditioned. In this study, we develop and implement a new approach to find a basis for the splitting preconditioner, based on standard rectangular LU factorization with partial permutation of the scaled transpose linear programming constraint matrix. In most cases, this basis is better conditioned than the existing one. In addition, we include a penalty parameter in Mehrotra’s predictor-corrector method in order to reduce ill-conditioning of the normal equations matrix. Computational experiments show a reduction in the average number of iterations of the preconditioned conjugate gradient method. Also, the increased efficiency and robustness of the new approach become evident by the performance profile.     

Circulant integral operators as preconditioners for Wiener-Hopf equations

In this paper, we study the solutions of finite-section Wiener-Hopf equations by the preconditioned conjugate gradient method. Our main aim is to give an easy and general scheme of constructing good circulant integral operators as preconditioners for such equations. The circulant integral operators are constructed from sequences of conjugate symmetric functions {C }. Letk(t) denote the kernel function of the Wiener-Hopf equation and be its Fourier transform. We prove that for sufficiently large if {C } is uniformly bounded on the real lineR and the convolution product of the Fourier transform ofC with uniformly onR, then the circulant preconditioned Wiener-Hopf operator will have a clustered spectrum. It follows that the conjugate gradient method, when applied to solving the preconditioned operator equation, converges superlinearly. Several circulant integral operators possessing the clustering and fast convergence properties are constructed explicitly. Numerical examples are also given to demonstrate the performance of different circulant integral operators as preconditioners for Wiener-Hopf operators.Research supported in part by HKRGC grant no. 221600070.     

Fastiterative methods for least squares estimations

Least squares estimations have been used extensively in many applications, e.g. system identification and signal prediction. When the stochastic process is stationary, the least squares estimators can be found by solving a Toeplitz or near-Toeplitz matrix system depending on the knowledge of the data statistics. In this paper, we employ the preconditioned conjugate gradient method with circulant preconditioners to solve such systems. Our proposed circulant preconditioners are derived from the spectral property of the given stationary process. In the case where the spectral density functions() of the process is known, we prove that ifs() is a positive continuous function, then the spectrum of the preconditioned system will be clustered around 1 and the method converges superlinearly. However, if the statistics of the process is unknown, then we prove that with probability 1, the spectrum of the preconditioned system is still clustered around 1 provided that large data samples are taken. For finite impulse response (FIR) system identification problems, our numerical results show that annth order least squares estimator can usually be obtained inO(n logn) operations whenO(n) data samples are used. Finally, we remark that our algorithm can be modified to suit the applications of recursive least squares computations with the proper use of sliding window method arising in signal processing applications.Research supported in part by HKRGC grant no. 221600070, ONR contract no. N00014-90-J-1695 and DOE grant no. DE-FG03-87ER25037.     

Limited memory preconditioners for symmetric indefinite problems with application to structural mechanics

This paper presents a class of limited memory preconditioners (LMP) for solving linear systems of equations with symmetric indefinite matrices and multiple right‐hand sides. These preconditioners based on limited memory quasi‐Newton formulas require a small number k of linearly independent vectors and may be used to improve an existing first‐level preconditioner. The contributions of the paper are threefold. First, we derive a formula to characterize the spectrum of the preconditioned operator. A spectral analysis of the preconditioned matrix shows that the eigenvalues are all real and that the LMP class is able to cluster at least k eigenvalues at 1. Secondly, we show that the eigenvalues of the preconditioned matrix enjoy interlacing properties with respect to the eigenvalues of the original matrix provided that the k linearly independent vectors have been prior projected onto the invariant subspaces associated with the eigenvalues of the original matrix in the open right and left half‐plane, respectively. Third, we focus on theoretical properties of the Ritz‐LMP variant, where Ritz information is used to determine the k vectors. Finally, we illustrate the numerical behaviour of the Ritz limited memory preconditioners on realistic applications in structural mechanics that require the solution of sequences of large‐scale symmetric saddle‐point systems. Numerical experiments show the relevance of the proposed preconditioner leading to a significant decrease in terms of computational operations when solving such sequences of linear systems. A saving of up to 43% in terms of computational effort is obtained on one of these applications. Copyright © 2016 John Wiley & Sons, Ltd.     

A note on preconditioned GMRES for solving singular linear systems

For solving a singular linear system Ax=b by GMRES, it is shown in the literature that if A is range-symmetric, then GMRES converges safely to a solution. In this paper we consider preconditioned GMRES for solving a singular linear system, we construct preconditioners by so-called proper splittings, which can ensure that the coefficient matrix of the preconditioned system is range-symmetric.     

Two new modified Gauss–Seidel methods for linear system with -matrices

In 2002, H. Kotakemori et al. proposed the modified Gauss–Seidel (MGS) method for solving the linear system with the preconditioner [H. Kotakemori, K. Harada, M. Morimoto, H. Niki, A comparison theorem for the iterative method with the preconditioner () J. Comput. Appl. Math. 145 (2002) 373–378]. Since this preconditioner is constructed by only the largest element on each row of the upper triangular part of the coefficient matrix, the preconditioning effect is not observed on the nth row. In the present paper, to deal with this drawback, we propose two new preconditioners. The convergence and comparison theorems of the modified Gauss–Seidel methods with these two preconditioners for solving the linear system are established. The convergence rates of the new proposed preconditioned methods are compared. In addition, numerical experiments are used to show the effectiveness of the new MGS methods.     

Block two-stage preconditioners

Linear systems of the form Ax = b, where the matrix A is symmetric and positive definite, often arise from the discretization of elliptic partial differential equations. A very successful method for solving these linear systems is the preconditioned conjugate gradient method. In this paper, we study parallel preconditioners for the conjugate gradient method based on the block two-stage iterative methods. Sufficient conditions for the validity of these preconditioners are given. Computational results of these preconditioned conjugate gradient methods on two parallel computing systems are presented.     

A note on the efficient solution of matrix pencil systems

Algorithms for solving matrix pencil systems of linear equations, of the form (A+B)x=c+d, are developed and analysed. The techniques that are discussed are based on methods for the generalized eigenvalue problem and avoid refactoring a matrix when the scalar changes. Numerical results are presented which demonstrate the advantages of the new techniques.The work of the first author was supported by the National Research Council of Canada and that of the second author by the National Science Foundation under Grant # MCS 76-24433.     

Truncated low‐rank methods for solving general linear matrix equations

This work is concerned with the numerical solution of large‐scale linear matrix equations . The most straightforward approach computes from the solution of an mn × mn linear system, typically limiting the feasible values of m,n to a few hundreds at most. Our new approach exploits the fact that X can often be well approximated by a low‐rank matrix. It combines greedy low‐rank techniques with Galerkin projection and preconditioned gradients. In turn, only linear systems of size m × m and n × n need to be solved. Moreover, these linear systems inherit the sparsity of the coefficient matrices, which allows to address linear matrix equations as large as m = n = O(10⁵). Numerical experiments demonstrate that the proposed methods perform well for generalized Lyapunov equations. Even for the case of standard Lyapunov equations, our methods can be advantageous, as we do not need to assume that C has low rank. Copyright © 2015 John Wiley & Sons, Ltd.