Additive preconditioning for matrix computations

Our randomized additive preconditioners are readily available and regularly facilitate the solution of linear systems of equations and eigen-solving for a very large class of input matrices. We study the generation of such preconditioners and their impact on the rank and the condition number of a matrix. We also propose some techniques for their refinement and two alternative versions of randomized preprocessing. Our analysis and experiments show the power of our approach even where we employ weak randomization, that is generate sparse and structured preconditioners, defined by a small number of random parameters.     

A tearing-based hybrid parallel banded linear system solver

A new parallel algorithm for the solution of banded linear systems is proposed. The scheme tears the coefficient matrix into several overlapped independent blocks in which the size of the overlap is equal to the system’s bandwidth. A corresponding splitting of the right-hand side is also provided. The resulting independent, and smaller size, linear systems are solved under the constraint that the solutions corresponding to the overlap regions are identical. This results in a linear system whose size is proportional to the sum of the overlap regions which we refer to as the “balance” system. We propose a solution strategy that does not require obtaining this “balance” system explicitly. Once the balance system is solved, retrieving the rest of the solution can be realized with almost perfect parallelism. Our proposed algorithm is a hybrid scheme that combines direct and iterative methods for solving a single banded system of linear equations on parallel architectures. It has broad applications in finite-element analysis, particularly as a parallel solver of banded preconditioners that can be used in conjunction with outer Krylov iterative schemes.     

Numerical study on incomplete orthogonal factorization preconditioners

We design, analyse and test a class of incomplete orthogonal factorization preconditioners constructed from Givens rotations, incorporating some dropping strategies and updating tricks, for the solution of large sparse systems of linear equations. Comprehensive accounts about how the preconditioners are coded, what storage is required and how the computation is executed for a given accuracy are presented. A number of numerical experiments show that these preconditioners are competitive with standard incomplete triangular factorization preconditioners when they are applied to accelerate Krylov subspace iteration methods such as GMRES and BiCGSTAB.     

The spectral properties of the preconditioned matrix for nonsymmetric saddle point problems

In this paper, on the basis of matrix splitting, two preconditioners are proposed and analyzed, for nonsymmetric saddle point problems. The spectral property of the preconditioned matrix is studied in detail. When the iteration parameter becomes small enough, the eigenvalues of the preconditioned matrices will gather into two clusters—one is near (0,0) and the other is near (2,0)—for the PPSS preconditioner no matter whether A is Hermitian or non-Hermitian and for the PHSS preconditioner when A is a Hermitian or real normal matrix. Numerical experiments are given, to illustrate the performances of the two preconditioners.     

Minimum residual methods for augmented systems

For large systems of linear equations, iterative methods provide attractive solution techniques. We describe the applicability and convergence of iterative methods of Krylov subspace type for an important class of symmetric and indefinite matrix problems, namely augmented (or KKT) systems. Specifically, we consider preconditioned minimum residual methods and discuss indefinite versus positive definite preconditioning. For a natural choice of starting vector we prove that when the definite and indenfinite preconditioners are related in the obvious way, MINRES (which is applicable in the case of positive definite preconditioning) and full GMRES (which is applicable in the case of indefinite preconditioning) give residual vectors with identical Euclidean norm at each iteration. Moreover, we show that the convergence of both methods is related to a system of normal equations for which the LSQR algorithm can be employed. As a side result, we give a rare example of a non-trivial normal(1) matrix where the corresponding inner product is explicitly known: a conjugate gradient method therefore exists and can be employed in this case. This work was supported by British Council/German Academic Exchange Service Research Collaboration Project 465 and NATO Collaborative Research Grant CRG 960782     

A numerical study of optimized sparse preconditioners

Preconditioning strategies based on incomplete factorizations and polynomial approximations are studied through extensive numerical experiments. We are concerned with the question of the optimal rate of convergence that can be achieved for these classes of preconditioners.Our conclusion is that the well-known Modified Incomplete Cholesky factorization (MIC), cf. e.g., Gustafsson [20], and the polynomial preconditioning based on the Chebyshev polynomials, cf. Johnson, Micchelli and Paul [22], have optimal order of convergence as applied to matrix systems derived by discretization of the Poisson equation. Thus for the discrete two-dimensional Poisson equation withn unknowns,O(n ^1/4) andO(n ^1/2) seem to be the optimal rates of convergence for the Conjugate Gradient (CG) method using incomplete factorizations and polynomial preconditioners, respectively. The results obtained for polynomial preconditioners are in agreement with the basic theory of CG, which implies that such preconditioners can not lead to improvement of the asymptotic convergence rate.By optimizing the preconditioners with respect to certain criteria, we observe a reduction of the number of CG iterations, but the rates of convergence remain unchanged.Supported by The Norwegian Research Council for Science and the Humanities (NAVF) under grants no. 413.90/002 and 412.93/005.Supported by The Royal Norwegian Council for Scientific and Industrial Research (NTNF) through program no. STP.28402: Toolkits in industrial mathematics.     

Preconditioners for spectral element methods for elliptic and parabolic problems

In this paper we propose preconditioners for spectral element methods for elliptic and parabolic problems. These preconditioners are constructed using separation of variables and are easy to invert. Moreover they are spectrally equivalent to the quadratic forms which they are used to approximate.     

BTTB preconditioners for BTTB least squares problems

In this paper, we consider solving the least squares problem min_x‖b-Tx‖₂ by using preconditioned conjugate gradient (PCG) methods, where T is a large rectangular matrix which consists of several square block-Toeplitz-Toeplitz-block (BTTB) matrices and b is a column vector. We propose a BTTB preconditioner to speed up the PCG method and prove that the BTTB preconditioner is a good preconditioner. We then discuss the construction of the BTTB preconditioner. Numerical examples, including image restoration problems, are given to illustrate the efficiency of our BTTB preconditioner. Numerical results show that our BTTB preconditioner is more efficient than the well-known Level-1 and Level-2 circulant preconditioners.     

Modified block preconditioners for the discretized time-harmonic Maxwell equations in mixed form

In this paper, building on the previous work by Greif and Schötzau [Preconditioners for the discretized time-harmonic Maxwell equations in mixed form, Numer. Linear Algebra Appl. 14 (2007) 281–297] and Benzi and Olshanskii [An augmented lagrangian-based approach to the Oseen problem, SIAM J. Sci. Comput. 28 (2006) 2095–2113], we present the improved preconditioning techniques for the iterative solution of the saddle point linear systems, which arise from the finite element discretization of the mixed formulation of the time-harmonic Maxwell equations. The modified block diagonal and triangular preconditioners considered are based on augmentation with using the symmetric nonsingular weighted matrix. We discuss the spectral properties of the preconditioned matrix in detail and generalize the results of the above-mentioned paper by Greif and Schötzau. Numerical experiments are given to demonstrate the efficiency of the presented preconditioners.     

ADI preconditioned Krylov methods for large Lyapunov matrix equations

In the present paper, we propose preconditioned Krylov methods for solving large Lyapunov matrix equations AX+XA^T+BB^T=0. Such problems appear in control theory, model reduction, circuit simulation and others. Using the Alternating Direction Implicit (ADI) iteration method, we transform the original Lyapunov equation to an equivalent symmetric Stein equation depending on some ADI parameters. We then define the Smith and the low rank ADI preconditioners. To solve the obtained Stein matrix equation, we apply the global Arnoldi method and get low rank approximate solutions. We give some theoretical results and report numerical tests to show the effectiveness of the proposed approaches.     

Local refinement techniques for elliptic problems on cell-centered grids

Summary Algebraic multilevel analogues of the BEPS preconditioner designed for solving discrete elliptic problems on grids with local refinement are formulated, and bounds on their relative condition numbers, with respect to the composite-grid matrix, are derived. TheV-cycle and, more generally,v-foldV-cycle multilevel BEPS preconditioners are presented and studied. It is proved that for 2-D problems theV-cycle multilevel BEPS is almost optimal, whereas thev-foldV-cycle algebraic multilevel BEPS is optimal under a mild restriction on the composite cell-centered grid. For thev-fold multilevel BEPS, the variational relation between the finite difference matrix and the corresponding matrix on the next-coarser level is not necessarily required. Since they are purely algebraically derived, thev-fold (v>1) multilevel BEPS preconditioners perform without any restrictionson the shape of subregions, unless the refinement is too fast. For theV-cycle BEPS preconditioner (2-D problem), a variational relation between the matrices on two consecutive grids is required, but there is no restriction on the method of refinement on the shape, or on the size of the subdomains.     

A unifying approach to abstract matrix algebra preconditioning

In earlier papers Tyrtyshnikov [42] and the first author [14] considered the analysis of clustering properties of the spectra of specific Toeplitz preconditioned matrices obtained by means of the best known matrix algebras. Here we generalize this technique to a generic Banach algebra of matrices by devising general preconditioners related to “convergent” approximation processes [36]. Finally, as case study, we focus our attention on the Tau preconditioning by showing how and why the best matrix algebra preconditioners for symmetric Toeplitz systems can be constructed in this class. Received April 25, 1997 / Revised version received March 13, 1998     

Two-level hierarchically preconditioned conjugate gradient methods for solving linear elasticity finite element equations

In this paper we study and compare some preconditioned conjugate gradient methods for solving large-scale higher-order finite element schemes approximating two- and three-dimensional linear elasticity boundary value problems. The preconditioners discussed in this paper are derived from hierarchical splitting of the finite element space first proposed by O. Axelsson and I. Gustafsson. We especially focus our attention to the implicit construction of preconditioning operators by means of some fixpoint iteration process including multigrid techniques. Many numerical experiments confirm the efficiency of these preconditioners in comparison with classical direct methods most frequently used in practice up to now.     

A framework for polynomial preconditioners based on fast transforms I: Theory

Optimal and superoptimal approximations of a complex square matrix by polynomials in a normal basis matrix are considered. If the unitary transform associated with the eigenvectors of the basis matrix is computable using a fast algorithm, the approximations may be utilized for constructing preconditioners. Theorems describing how the parameters of the approximations could be efficiently computed are given, and for special cases earlier results by other authors are recovered. Also, optimal and superoptimal approximations for block matrices are determined, and the same type of theorems as for the point case are proved. This research was supported by the Swedish National Board for Industrial and Technical Development (NUTEK) and by the U.S. National Science Foundation under grant ASC-8958544.     

A framework for polynomial preconditioners based on fast transforms II: PDE applications

The solution of systems of equations arising from systems of time-dependent partial differential equations (PDEs) is considered. Primarily, first-order PDEs are studied, but second-order derivatives are also accounted for. The discretization is performed using a general finite difference stencil in space and an implicit method in time. The systems of equations are solved by a preconditioned Krylov subspace method. The preconditioners exploit optimal and superoptimal approximations by low-degree polynomials in a normal basis matrix, associated with a fast trigonometric transform. Numerical experiments for high-order accurate discretizations are presented. The results show that preconditioners based on fast transforms yield efficient solution algorithms, even for large quotients between the time and space steps. Utilizing a spatial grid ratio less than one, the arithmetic work per grid point is bounded by a constant as the number of grid points increases. This research was supported by the Swedish National Board for Industrial and Technical Development (NUTEK) and by the U.S. National Science Foundation under grant ASC-8958544.     

On the eigenvalues of a class of saddle point matrices

We study spectral properties of a class of block 2 × 2 matrices that arise in the solution of saddle point problems. These matrices are obtained by a sign change in the second block equation of the symmetric saddle point linear system. We give conditions for having a (positive) real spectrum and for ensuring diagonalizability of the matrix. In particular, we show that these properties hold for the discrete Stokes operator, and we discuss the implications of our characterization for augmented Lagrangian formulations, for Krylov subspace solvers and for certain types of preconditioners. The work of this author was supported in part by the National Science Foundation grant DMS-0207599 Revision dated 5 December 2005.     

Multilevel additive and multiplicative methods for orthogonal spline collocation problems

Summary. Multilevel preconditioners are proposed for the iterative solution of the discrete problems which arise when orthogonal spline collocation with piecewise Hermite bicubics is applied to the Dirichlet boundary value problem for a self-adjoint elliptic partial differential equation on a rectangle. Additive and multiplicative preconditioners are defined respectively as sums and products of independent operators on a sequence of nested subspaces of the fine partition approximation space. A general theory of additive and multiplicative Schwarz methods is used to prove that the preconditioners are spectrally equivalent to the collocation discretization of the Laplacian with the spectral constants independent of the fine partition stepsize and the number of levels. The preconditioned conjugate gradient and preconditioned Orthomin methods are considered for the solution of collocation problems. An implementation of the methods is discussed and the results of numerical experiments are presented. Received March 1, 1994 / Revised version received January 23, 1996     

A dual iterative substructuring method with a penalty term

An iterative substructuring method with Lagrange multipliers is considered for second order elliptic problems, which is a variant of the FETI-DP method. The standard FETI-DP formulation is associated with the saddle-point problem which is induced from the minimization problem with a constraint for imposing the continuity across the interface. Starting from the slightly changed saddle-point problem by addition of a penalty term with a positive penalization parameter η, we propose a dual substructuring method which is implemented iteratively by the conjugate gradient method. In spite of the absence of any preconditioners, it is shown that the proposed method is numerically scalable in the sense that for a large value of η, the condition number of the resultant dual problem is bounded by a constant independent of both the subdomain size H and the mesh size h. Computational issues and numerical results are presented. This work was partially supported by the SRC/ERC program of MOST/KOSEF(R11-2002-103).     

On block-circulant preconditioners for high-order compact approximations of convection-diffusion problems

We study some properties of block-circulant preconditioners for high-order compact approximations of convection-diffusion problems. For two-dimensional problems, the approximation gives rise to a nine-point discretisation matrix and in three dimensions, we obtain a nineteen-point matrix. We derive analytical expressions for the eigenvalues of the block-circulant preconditioner and this allows us to establish the invertibility of the preconditioner in both two and three dimensions. The eigenspectra of the preconditioned matrix in the two-dimensional case is described for different test cases. Our numerical results indicate that the block-circulant preconditioning leads to significant reduction in iteration counts and comparisons between the high-order compact and upwind discretisations are carried out. For the unpreconditioned systems, we observe fewer iteration counts for the HOC discretisation but for the preconditioned systems, we find similar iteration counts for both finite difference approximations of constant-coefficient two-dimensional convection-diffusion problems.     

Substructuring preconditioners for saddle-point problems arising from Maxwell's equations in three dimensions

This paper is concerned with the saddle-point problems arising from edge element discretizations of Maxwell's equations in a general three dimensional nonconvex polyhedral domain. A new augmented technique is first introduced to transform the problems into equivalent augmented saddle-point systems so that they can be solved by some existing preconditioned iterative methods. Then some substructuring preconditioners are proposed, with very simple coarse solvers, for the augmented saddle-point systems. With the preconditioners, the condition numbers of the preconditioned systems are nearly optimal; namely, they grow only as the logarithm of the ratio between the subdomain diameter and the finite element mesh size.