Combination of numerical and structured approaches to the construction of a second-order incomplete triangular factorization in parallel preconditioning methods

Parallel versions of the stabilized second-order incomplete triangular factorization conjugate gradient method in which the reordering of the coefficient matrix corresponding to the ordering based on splitting into subdomains with separators are considered. The incomplete triangular factorization is organized using the truncation of fill-in “by value” at internal nodes of subdomains, and “by value” and ‘by positions” on the separators. This approach is generalized for the case of constructing a parallel version of preconditioning the second-order incomplete LU factorization for nonsymmetric diagonally dominant matrices with. The reliability and convergence rate of the proposed parallel methods is analyzed. The proposed algorithms are implemented using MPI, results of solving benchmark problems with matrices from the collection of the University of Florida are presented.     

Displacement decomposition—incomplete factorization preconditioning techniques for linear elasticity problems

Two preconditioning techniques for the numerical solution of linear elasticity problems are described and studied. Both techniques are based on spectral equivalence approach. The first technique consists in an incomplete factorization of the separate displacement component part of the stiffness matrix. The second technique uses an incomplete factorization of the isotropic approximation to the stiffness matrix. Results concerning existence, stability and efficiency of these preconditioning techniques are presented. The efficiency and robustness of the described techniques are illustrated by numerical experiments.     

DRIC: A dynamic version of the RIC method

A new incomplete factorization method is proposed, differing from previous ones by the way in which the diagonal entries of the triangular factors are defined. A comparison is given with the dynamic modified incomplete factorization methods of Axelsson–Barker and Beauwens, and with the relaxed incomplete Cholesky method of Axelsson and Lindskog. Theoretical arguments show that the new method is at least as robust as both previous ones, while numerical experiments made in the discrete PDE context show an effective improvement in many practical circumstances, particularly for anisotropic problems.     

Diagonal threshold techniques in robust multi‐level ILU preconditioners for general sparse linear systems

This paper introduces techniques based on diagonal threshold tolerance when developing multi‐elimination and multi‐level incomplete LU (ILUM) factorization preconditioners for solving general sparse linear systems. Existing heuristics solely based on the adjacency graph of the matrices have been used to find independent sets and are not robust for matrices arising from certain applications in which the matrices may have small or zero diagonals. New heuristic strategies based on the adjacency graph and the diagonal values of the matrices for finding independent sets are introduced. Analytical bounds for the factorization and preconditioned errors are obtained for the case of a two‐level analysis. These bounds provide useful information in designing robust ILUM preconditioners. Extensive numerical experiments are conducted in order to compare robustness and efficiency of various heuristic strategies. Copyright © 1999 John Wiley & Sons, Ltd.     

The importance of structure in incomplete factorization preconditioners

In this paper, we consider level-based preconditioning, which is one of the basic approaches to incomplete factorization preconditioning of iterative methods. It is well-known that while structure-based preconditioners can be very useful, excessive memory demands can limit their usefulness. Here we present an improved strategy that considers the individual entries of the system matrix and restricts small entries to contributing to fewer levels of fill than the largest entries. Using symmetric positive-definite problems arising from a wide range of practical applications, we show that the use of variable levels of fill can yield incomplete Cholesky factorization preconditioners that are more efficient than those resulting from the standard level-based approach. The concept of level-based preconditioning, which is based on the structural properties of the system matrix, is then transferred to the numerical incomplete decomposition. In particular, the structure of the incomplete factorization determined in the symbolic factorization phase is explicitly used in the numerical factorization phase. Further numerical results demonstrate that our level-based approach can lead to much sparser but efficient incomplete factorization preconditioners.     

The effect of ordering on preconditioned conjugate gradients

We investigate the effect of the ordering of the unknowns on the convergence of the preconditioned conjugate gradient method. We examine a wide range of ordering methods including nested dissection, minimum degree, and red-black and consider preconditionings without fill-in. We show empirically that there can be a significant difference in the number of iterations required by the conjugate gradient method and suggest reasons for this marked difference in performance.We also consider the effect of orderings when an incomplete factorization which allows some fill-in is performed. We consider the effect of automatically controlling the sparsity of the incomplete factorization through drop tolerances and level of fill-in.     

ILUT: A dual threshold incomplete LU factorization

In this paper we describe an Incomplete LU factorization technique based on a strategy which combines two heuristics. This ILUT factorization extends the usual ILU(O) factorization without using the concept of level of fill-in. There are two traditional ways of developing incomplete factorization preconditioners. The first uses a symbolic factorization approach in which a level of fill is attributed to each fill-in element using only the graph of the matrix. Then each fill-in that is introduced is dropped whenever its level of fill exceeds a certain threshold. The second class of methods consists of techniques derived from modifications of a given direct solver by including a dropoff rule, based on the numerical size of the fill-ins introduced, traditionally referred to as threshold preconditioners. The first type of approach may not be reliable for indefinite problems, since it does not consider numerical values. The second is often far more expensive than the standard ILU(O). The strategy we propose is a compromise between these two extremes.     

A class of multilevel recursive incomplete LU preconditioning techniques

We introduce a class of multilevel recursive incomplete LU preconditioning techniques (RILUM) for solving general sparse matrices. This technique is based on a recursive two by two block incomplete LU factorization on the coefficient matrix. The coarse level system is constructed as an (approximate) Schur complement. A dynamic preconditioner is obtained by solving the Schur complement matrix approximately. The novelty of the proposed techniques is to solve the Schur complement matrix by a preconditioned Krylov subspace method. Such a reduction process is repeated to yield a multilevel recursive preconditioner.     

Linearized factorization techniques for multidimensional reaction—diffusion equations

An iterative predictor—corrector technique for the elimination of the approximate factorization errors which result from the factorization of linearized θ-methods in multidimensional reaction—diffusion equations is proposed, and its convergence and linear stability are analyzed. Four approximate factorization techniques which do not account for the approximate factorization errors are developed. The first technique uses the full Jacobian matrix of the reaction terms, requires the inversion of, in general, dense matrices, and its approximate factorization errors are second-order accurate in time. The second and third methods approximate the Jacobian matrix by diagonal or triangular ones which are easily inverted but their approximate factorization errors are, however, first-order accurate in time. The fourth approximately factorized method has approximate factorization errors which are second-order accurate in time and requires the inversion of lower and upper triangular matrices. The techniques are applied to a nonlinear, two-species, two-dimensional system of reaction—diffusion equations in order to determine the approximate factorization errors and those resulting from the approximations to the Jacobian matrix as functions of the allocation of the reaction terms, space and time.     

Heuristic Spectral Techniques for the Reduction of Bandwidth and Work-Bound of Sparse Matrices

In this paper, we present heuristic techniques for the reduction of the bandwidth of a sparse matrix as well as for the reduction of the cost of the associated Cholesky factorization. Our algorithms are inspired by the spectral method of Barnard, Pothen and Simon (1995), which derives a permutation for reducing the envelope-size of a sparse matrix by computing the second eigenvector of the associated Laplacian matrix. Two main modifications of that method are proposed and tested. The first is based on the experimental observation that it is often preferable to perform only few iterations of an iterative method converging to the second eigenvector; the second is the introduction of a weighted Laplacian. These simple ideas allow us to obtain a family of spectral methods that have been carefully tested on a set of matrices whose size ranges from few hundred to one million.     

Spectral analysis of parallel incomplete factorizations with implicit pseudo‐overlap

Two general parallel incomplete factorization strategies are investigated. The techniques may be interpreted as generalized domain decomposition methods. In contrast to classical domain decomposition methods, adjacent subdomains exchange data during the construction of the incomplete factorization matrix, as well as during each local forward elimination and each local backward elimination involved in the application of the preconditioner. Local renumberings of nodes are combined with suitable global fill‐in strategy in an (successful) attempt to overcome the well‐known trade‐off between high parallelism (locality) and fast convergence (globality). From an algebraic viewpoint, our techniques may be implemented as global renumbering strategies. Theoretical spectral analysis is provided, which displays that the convergence rate weakly depends on the number of subdomains. Numerical results obtained on a 16‐processor SGI Origin 2000 are reported, showing the efficiency of our parallel preconditionings. Copyright © 2001 John Wiley & Sons, Ltd.     

Updating incomplete factorization preconditioners for model order reduction

When solving a sequence of related linear systems by iterative methods, it is common to reuse the preconditioner for several systems, and then to recompute the preconditioner when the matrix has changed significantly. Rather than recomputing the preconditioner from scratch, it is potentially more efficient to update the previous preconditioner. Unfortunately, it is not always known how to update a preconditioner, for example, when the preconditioner is an incomplete factorization. A recently proposed iterative algorithm for computing incomplete factorizations, however, is able to exploit an initial guess, unlike existing algorithms for incomplete factorizations. By treating a previous factorization as an initial guess to this algorithm, an incomplete factorization may thus be updated. We use a sequence of problems from model order reduction. Experimental results using an optimized GPU implementation show that updating a previous factorization can be inexpensive and effective, making solving sequences of linear systems a potential niche problem for the iterative incomplete factorization algorithm.     

A modified Newton method for minimization

Some promising ideas for minimizing a nonlinear function, whose first and second derivatives are given, by a modified Newton method, were introduced by Fiacco and McCormick (Ref. 1). Unfortunately, in developing a method around these ideas, Fiacco and McCormick used a potentially unstable, or even impossible, matrix factorization. Using some recently developed techniques for factorizing an indefinite symmetric matrix, we are able to produce a method which is similar to Fiacco and McCormick's original method, but avoids the difficulties of the original method.Both authors gratefully acknowledge the award of a research fellowship from the British Science Research Council.     

A variant of block incomplete factorization preconditioners for a symmetricH-matrix

We propose a variant of parallel block incomplete factorization preconditioners for a symmetric block-tridiagonalH-matrix. Theoretical properties of these block preconditioners are compared with those of block incomplete factorization preconditioners for the corresponding comparison matrix. Numerical results of the preconditioned CG(PCG) method using these block preconditioners are compared with those of PCG using other types of block incomplete factorization preconditioners. Lastly, parallel computations of the block incomplete factorization preconditioners are carried out on the Cray C90.     

Overlapping for preconditioners based on incomplete factorizations and nested arrow form

In this paper, we discuss the usage of overlapping techniques for improving the convergence of preconditioners based on incomplete factorizations. To enable parallelism, these preconditioners are usually applied after the input matrix is permuted into a nested arrow form using k‐way nested dissection. This graph partitioning technique uses k‐way partitionning by vertex separator to recursively partition the graph of the input matrix into k subgraphs using a subset of its vertices called a separator. The overlapping technique is then based on algebraically extending the associated subdomains of these subgraphs and their corresponding separators obtained from k‐way nested dissection by their direct neighbours. A similar approach is known to accelerate the convergence of domain decomposition methods, where the input matrix is partitioned into a number of independent subdomains using k‐way vertex partitioning of a graph by edge separators, a different graph decomposition technique. We discuss the effect of the overlapping technique on the convergence of two classes of preconditioners, on the basis of nested factorization and block incomplete LDU factorization. Copyright © 2014 John Wiley & Sons, Ltd.     

A multilevel Crout ILU preconditioner with pivoting and row permutation

In this paper, we present a new incomplete LU factorization using pivoting by columns and row permutation. Pivoting by columns helps to avoid small pivots and row permutation is used to promote sparsity. This factorization is used in a multilevel framework as a preconditioner for iterative methods for solving sparse linear systems. In most multilevel incomplete ILU factorization preconditioners, preprocessing (scaling and permutation of rows and columns of the coefficient matrix) results in further improvements. Numerical results illustrate that these preconditioners are suitable for a wide variety of applications. Copyright © 2007 John Wiley & Sons, Ltd.     

A multilevel block incomplete cholesky preconditioner for solving normal equations in linear least squares problems

An incomplete factorization method for preconditioning symmetric positive definite matrices is introduced to solve normal equations. The normal equations are form to solve linear least squares problems. The procedure is based on a block incomplete Cholesky factorization and a multilevel recursive strategy with an approximate Schur complement matrix formed implicitly. A diagonal perturbation strategy is implemented to enhance factorization robustness. The factors obtained are used as a preconditioner for the conjugate gradient method. Numerical experiments are used to show the robustness and efficiency of this preconditioning technique, and to compare it with two other preconditioners.     

Relaxed and stabilized incomplete factorizations for non-self-adjoint linear systems

Two classes of incomplete factorization preconditioners are considered for nonsymmetric linear systems arising from second order finite difference discretizations of non-self-adjoint elliptic partial differential equations. Analytic and experimental results show that relaxed incomplete factorization methods exhibit numerical instabilities of the type observed with other incomplete factorizations, and the effects of instability are characterized in terms of the relaxation parameter. Several stabilized incomplete factorizations are introduced that are designed to avoid numerically unstable computations. In experiments with two-dimensional problems with variable coefficients and on nonuniform meshes, the stabilized methods are shown to be much more robust than standard incomplete factorizations.The work presented in this paper was supported by the National Science Foundation under grants DMS-8607478, CCR-8818340, and ASC-8958544, and by the U.S. Army Research Office under grant DAAL-0389-K-0016.     

New factorization codes for sparse,symmetric and positive definite matrices

Suitable techniques for storing the matrix pattern during the factorization of sparse, symmetric and positive definite matrices are considered. Especially we discuss the consequences of switching from a sparse factorization code to a full code when the uneliminated part of the matrix is full or almost full. The resulting codes seem to be among the most efficient for solving one-off problems regarding both execution time and storage requirements.This work has been supported by the Danish Natural Science Research Council, Grant No. 511-8476.