Massively parallel preconditioners for symmetric positive definite linear systems

In this paper, we address the problem of solving sparse symmetric linear systems on parallel computers. With further restrictive assumptions on the matrix (e.g., bidiagonal or tridiagonal structure), several direct methods may be used. These methods give ideas for constructing efficient data parallel preconditioners for general positive definite symmetric matrices. We describe two examples of such preconditioners for which the factorization (i.e., the construction of the preconditioning matrix) turns out to be parallel. This revised version was published online in June 2006 with corrections to the Cover Date.     

Recent computational developments in Krylov subspace methods for linear systems

Many advances in the development of Krylov subspace methods for the iterative solution of linear systems during the last decade and a half are reviewed. These new developments include different versions of restarted, augmented, deflated, flexible, nested, and inexact methods. Also reviewed are methods specifically tailored to systems with special properties such as special forms of symmetry and those depending on one or more parameters. Copyright © 2006 John Wiley & Sons, Ltd.     

Dynamic block GMRES: an iterative method for block linear systems

We present variants of the block-GMRES() algorithms due to Vital and the block-LGMRES(,) by Baker, Dennis and Jessup, obtained with replacing the standard QR factorization by a rank-revealing QR factorization in the Arnoldi process. The resulting algorithm allows for dynamic block deflation whenever there is a linear dependency between the Krylov vectors or the convergence of a right-hand-side occurs. implementations of the algorithms were tested on a number of test matrices and the results show that in some cases a substantial reduction of the execution time is obtained. Also a parallel implementation of our variant of the block-GMRES() algorithm, using and was tested on parallel computer, showing good parallel efficiency. This work was carried out while the author was at IM/UFRGS.     

A note on simultaneous preconditioning and symmetrization of non‐symmetric linear systems

Motivated by the theory of self‐duality that provides a variational formulation and resolution for non‐self‐adjoint partial differential equations (Ann. Inst. Henri Poincaré (C) Anal Non Linéaire 2007; 24 :171–205; Selfdual Partial Differential Systems and Their Variational Principles. Springer: New York, 2008), we propose new templates for solving large non‐symmetric linear systems. The method consists of combining a new scheme that simultaneously preconditions and symmetrizes the problem, with various well‐known iterative methods for solving linear and symmetric problems. The approach seems to be efficient when dealing with certain ill‐conditioned, and highly non‐symmetric systems. The numerical and theoretical results are provided to show the efficiency of our approach. Copyright © 2010 John Wiley & Sons, Ltd.     

Flexible and multi‐shift induced dimension reduction algorithms for solving large sparse linear systems

We give two generalizations of the induced dimension reduction (IDR) approach for the solution of linear systems. We derive a flexible and a multi‐shift quasi‐minimal residual IDR variant. These variants are based on a generalized Hessenberg decomposition. We present a new, more stable way to compute basis vectors in IDR. Numerical examples are presented to show the effectiveness of these new IDR variants and the new basis compared with existing ones and to other Krylov subspace methods. Copyright © 2014 John Wiley & Sons, Ltd.     

The use of second degree normalized implicit conjugate gradient methods for solving large sparse systems of linear equations

Second degree normalized implicit conjugate gradient methods for the numerical solution of self-adjoint elliptic partial differential equations are developed. A proposal for the selection of certain values of the iteration parameters ?_i, γ_i involved in solving two and three-dimensional elliptic boundary-value problems leading to substantial savings in computational work is presented. Experimental results for model problems are given.     

Weighted FOM and GMRES for solving nonsymmetric linear systems

This paper presents two new methods called WFOM and WGMRES, which are variants of FOM and GMRES, for solving large and sparse nonsymmetric linear systems. To accelerate the convergence, these new methods use a different inner product instead of the Euclidean one. Furthermore, at each restart, a different inner product is chosen. The weighted Arnoldi process is introduced for implementing these methods. After describing the weighted methods, we give the relations that link them to FOM and GMRES. Experimental results are presented to show the good performances of the new methods compared to FOM(m) and GMRES(m). This revised version was published online in August 2006 with corrections to the Cover Date.     

Almost block diagonal linear systems: sequential and parallel solution techniques,and applications

Almost block diagonal (ABD) linear systems arise in a variety of contexts, specifically in numerical methods for two‐point boundary value problems for ordinary differential equations and in related partial differential equation problems. The stable, efficient sequential solution of ABDs has received much attention over the last fifteen years and the parallel solution more recently. We survey the fields of application with emphasis on how ABDs and bordered ABDs (BABDs) arise. We outline most known direct solution techniques, both sequential and parallel, and discuss the comparative efficiency of the parallel methods. Finally, we examine parallel iterative methods for solving BABD systems. Copyright © 2000 John Wiley & Sons, Ltd.     

On the convergence behavior of the restarted GMRES algorithm for solving nonsymmetric linear systems

The solution of nonsymmetric systems of linear equations continues to be a difficult problem. A main algorithm for solving nonsymmetric problems is restarted GMRES. The algorithm is based on restarting full GMRES every s iterations, for some integer s>0. This paper considers the impact of the restart frequency s on the convergence and work requirements of the method. It is shown that a good choice of this parameter can lead to reduced solution time, while an improper choice may hinder or preclude convergence. An adaptive procedure is also presented for determining automatically when to restart. The results of numerical experiments are presented.     

A parallel algorithm for large systems of Volterra integral equations of Abel type

A significative number of recent applications require numerical solution of large systems of Abel–Volterra integral equations. Here we propose a parallel algorithm to numerically solve a class of these systems, designed for a distributed-memory MIMD architecture. In order to achieve a good efficiency we employ a fully parallel and fast convergent waveform relaxation (WR) method and evaluate the lag term by using FFT techniques. To accelerate the convergence of the WR method and to best exploit the parallel architecture we develop special strategies. The performances of the resulting code, NSWR4, are illustrated on some examples.     

Subspace‐by‐subspace preconditioners for structured linear systems

We consider the iterative solution of symmetric positive‐definite linear systems whose coefficient matrix may be expressed as the outer product of low‐rank terms. We derive suitable preconditioners for such systems, and demonstrate their effectiveness on a number of test examples. We also consider combining these methods with existing techniques to cope with the commonly‐occuring case where the coefficient matrix is the linear sum of elements, some of which are of very low rank. Copyright © 1999 John Wiley & Sons, Ltd.     

On a parallel multilevel solver for linear elasticity problems

In this paper an application of the additive multilevel iteration method to parallel solving of large‐scale linear elasticity problems is considered. The results are derived in the framework of the hierarchical basis finite element discretization defined on a tensor product of one‐dimensional grid and a sequence of nested triangulations . The algorithm was tested on a number of model problems, arising from bridge foundation modeling. Parallel performance of the solver is reported for Cray T3E‐600 and Sun ES/4000 computer systems. Copyright © 1999 John Wiley & Sons, Ltd.     

Numerical solution of parameter‐dependent linear systems

We are interested in the numerical solution of the complex large linear system, (σ²A+σB+C)x=f(σ), for many, possibly a few hundreds, values of the complex parameter σ in a wide range. We assume that A, B and C are large, sparse, symmetric matrices, as is the case in several application problems. In particular, we focus on the following structured right‐hand side, f(σ)=FΦ(σ), where F is a (tall) rectangular matrix whose entries are independent of σ. We propose to approximate the solution x=x(σ) by means of a projection onto a single vector subspace, and a subsequent solution of the reduced dimension problem, for all values of interest of the parameter σ. Numerical experiments report the effectiveness of our approach on real application problems. Copyright © 2005 John Wiley & Sons, Ltd.     

A restarted Krylov method with inexact inversions

In this paper, we present a new type of restarted Krylov method for calculating the smallest eigenvalues of a symmetric positive definite matrix G. The new framework avoids the Lanczos tridiagonalization process and the use of polynomial filtering. This simplifies the restarting mechanism and allows the introduction of several modifications. Convergence is assured by a monotonicity property that pushes the eigenvalues toward their limits. Another innovation is the use of inexact inversions of G to generate the Krylov matrices. In this approach, the inverse of G is approximated by using an iterative method to solve the related linear system. Numerical experiments illustrate the usefulness of the proposed approach.     

Block truncated-Newton methods for parallel optimization

Truncated-Newton methods are a class of optimization methods suitable for large scale problems. At each iteration, a search direction is obtained by approximately solving the Newton equations using an iterative method. In this way, matrix costs and second-derivative calculations are avoided, hence removing the major drawbacks of Newton's method. In this form, the algorithms are well-suited for vectorization. Further improvements in performance are sought by using block iterative methods for computing the search direction. In particular, conjugate-gradient-type methods are considered. Computational experience on a hypercube computer is reported, indicating that on some problems the improvements in performance can be better than that attributable to parallelism alone.Partially supported by Air Force Office of Scientific Research grant AFOSR-85-0222.Partially supported by National Science Foundation grant ECS-8709795, co-funded by the U.S. Air Force Office of Scientific Research.     

Numerical experiments with ABS algorithms for linear systems on a parallel machine

Numerical results are obtained on sequential and parallel versions of ABS algorithms for linear systems for both full matrices andq-band matrices. The results using the sequential algorithm on full matrices indicate the superiority of a particular implementation of the symmetric algorithm. The condensed form of the algorithm is well suited for implementation in a parallel environment, and results obtained on the IBM 4381 system favor a synchronous implementation over the asynchronous one. Results are obtained from sequential implementations of theLU, Cholesky, and symmetric algorithms of the ABS class forq-band matrices able to reduce memory storage. A simple parallelization of do-loops for calculating components gives interesting performances.This work has been developed in the framework of a collaboration between IBM-ECSEC, Rome, Italy, and the Department of Mathematics of the University of Bergamo, Bergamo, Italy.The author is grateful to Prof. J. Abaffy (University of Economics, Budapest), Prof. L. Dixon (Hatfield Polytechnic), and Prof. E. Spedicato (Department of Mathematics, University of Bergamo) for useful suggestions.     

An EM-based iterative method for solving large sparse linear systems

ABSTRACT

We propose a novel iterative algorithm for solving a large sparse linear system. The method is based on the EM algorithm. If the system has a unique solution, the algorithm guarantees convergence with a geometric rate. Otherwise, convergence to a minimal Kullback–Leibler divergence point is guaranteed. The algorithm is easy to code and competitive with other iterative algorithms.     

CMRH: A new method for solving nonsymmetric linear systems based on the Hessenberg reduction algorithm

The Generalized Minimal Residual (GMRES) method and the Quasi-Minimal Residual (QMR) method are two Krylov methods for solving linear systems. The main difference between these methods is the generation of the basis vectors for the Krylov subspace. The GMRES method uses the Arnoldi process while QMR uses the Lanczos algorithm for constructing a basis of the Krylov subspace. In this paper we give a new method similar to QMR but based on the Hessenberg process instead of the Lanczos process. We call the new method the CMRH method. The CMRH method is less expensive and requires slightly less storage than GMRES. Numerical experiments suggest that it has behaviour similar to GMRES. This revised version was published online in June 2006 with corrections to the Cover Date.     

A note on ℝ‐linear GMRES for solving a class of ℝ‐linear systems

We prove that ?‐linear GMRES for solving a class of ?‐linear systems is faster than GMRES applied to the related ?‐linear systems in terms of matrix–vector products. Numerical examples are given to demonstrate the theoretical result. Copyright © 2011 John Wiley & Sons, Ltd.