Parallel algorithms for solving large linear systems

The solution of linear systems continues to play an important role in scientific computing. The problems to be solved often are of very large size, so that solving them requires large computer resources. To solve these problems, at least supercomputers with large shared memory or massive parallel computer systems with distributed memory are needed.

This paper gives a survey of research on parallel implementation of various direct methods to solve dense linear systems. In particular are considered: Gaussian elimination, Gauss-Jordan elimination and a variant due to Huard (1979), and an algorithm due to Enright (1978), designed in relation to solving (stiff) ODEs, such that stepsize and other method parameters can easily be varied.

Some theoretical results are mentioned, including a new result on error analysis of Huard's algorithm. Moreover, practical considerations and results of experiments on supercomputers and on a distributed-memory computer system are presented.     

Iterative algorithms for inverse-strongly accretive mappings with applications

The purpose of this paper is to consider an iterative method for finding solutions to a variational inequality for inverse-strongly accretive mappings. Strong convergence theorems, which are an analogue of weak convergence theorems proved by Aoyama, Iiduka and Takahashi (Fixed Point Theory Appl. 2006, 35390, 2006), are established in the framework of Banach spaces.     

Iterative algorithms for variational inequality and equilibrium problems with applications

In this paper, we introduce an iterative method for finding a common element of the set of solutions of equilibrium problems, of the set of variational inequalities and of the set of common fixed points of an infinite family of nonexpansive mappings in the framework of real Hilbert spaces. Strong convergence of the proposed iterative algorithm is obtained. As an application, we utilize the main results which improve the corresponding results announced in Chang et al. (Nonlinear Anal, 70:3307–3319, 2009), Colao et al. (J Math Anal Appl, 344:340–352, 2008), Plubtieng and Punpaeng (Appl Math Comput, 197:548–558, 2008) to study the optimization problem.     

Error estimates for linear systems with applications to regularization

In this paper, we discuss several (old and new) estimates for the norm of the error in the solution of systems of linear equations, and we study their properties. Then, these estimates are used for approximating the optimal value of the regularization parameter in Tikhonov’s method for ill-conditioned systems. They are also used as a stopping criterion in iterative methods, such as the conjugate gradient algorithm, which have a regularizing effect. Several numerical experiments and comparisons with other procedures show the effectiveness of our estimates.     

Iterative refinement for linear systems and LAPACK

E-mail: na.nhigham{at}na-net.ornl.gov The technique of iterative refinement for improving the computedsolution to a linear system was used on desk calculators andcomputers in the 1940s and has remained popular. In the 1990siterative refinement is well supported in software libraries,notably in LAPACK. Although the behaviour of iterative refinementin floating point arithmetic is reasonably well understood,the existing theory is not sufficient to justify the use offixed precision iterative refinement in all the LAPACK routinesin which it is implemented. We present analysis that providesthe theoretical support needed for LAPACK. The analysis coversboth mixed and fixed precision iterative refinement with anarbitrary number of iterations, makes only a general assumptionon the underlying solver, and is relatively short. We identifysome remaining open problems.     

Iterative regularization algorithms for constrained image deblurring on graphics processors

The ability of the modern graphics processors to operate on large matrices in parallel can be exploited for solving constrained image deblurring problems in a short time. In particular, in this paper we propose the parallel implementation of two iterative regularization methods: the well known expectation maximization algorithm and a recent scaled gradient projection method. The main differences between the considered approaches and their impact on the parallel implementations are discussed. The effectiveness of the parallel schemes and the speedups over standard CPU implementations are evaluated on test problems arising from astronomical images.     

Symmetric orthogonal approximation to symmetric tensors with applications to image reconstruction

The main objective of this paper is to study an approximation of symmetric tensors by symmetric orthogonal decomposition. We propose and study an iterative algorithm to determine a symmetric orthogonal approximation and analyze the convergence of the proposed algorithm. Numerical examples are reported to demonstrate the effectiveness of the proposed algorithm. We also apply the proposed algorithm to represent correlated face images. We demonstrate better face image reconstruction results by combining principal components and symmetric orthogonal approximation instead of combining principal components and higher‐order SVD results.     

Eigenstructure assignment for linear parameter-varying systems with applications

The aim of this paper is to show that a recently proposed technique for eigenstructure assignment of linear time-invariant systems can be extended to solve the corresponding eigenstructure assignment problem for linear parameter-varying systems, whose state-space matrices depend on a set of time-varying parameters that are bounded and available online. In particular, the design of eigenstructure assignment is performed without requiring any conditions on the closed-loop eigenvalues, and provides a simple, complete and analytical parametric approach as well as the most degrees of design freedom for the eigenstructure assignment problem of linear parameter-varying systems. A parameter-varying attitude control system of refueling spacecraft in-orbit is used to demonstrate the usefulness and practicality of the proposed approach.     

Iterative refinement for linear systems in variable-precision arithmetic

We define a binary-cascade iterative-refinement process (BCIR) for solving a nonsingular linear system with prescribed relative accuracy. For very high accuracy requirements BCIR uses a software-implemented variable-precision arithmetic. The time-cost and accuracy of BCIR are deduced under various conditions.     

Transpose-free Lanczos-type algorithms for nonsymmetric linear systems

The method of Lanczos for solving systems of linear equations is implemented by using recurrence relationships between formal orthogonal polynomials. A drawback is that the computation of the coefficients of these recurrence relationships usually requires the use of the transpose of the matrix of the system. Due to the indirect addressing, this is a costly operation. In this paper, a new procedure for computing these coefficients is proposed. It is based on the recursive computation of the products of polynomials appearing in their expressions and it does not involve the transpose of the matrix. Moreover, our approach allows to implement simultaneously and at a low price a Lanczos-type product method such as the CGS or the BiCGSTAB. This revised version was published online in June 2006 with corrections to the Cover Date.     

Decomposition method in correction problems for inconsistent systems of linear inequalities with partitioned matrices

Correction problems for inconsistent systems of linear inequalities with partitioned matrices are examined. The criteria used are based on the minimum of the sum of squares and the minimax weighted Euclidean norm of a block in the correction matrix.     

Robust inversion algorithms for vector linear systems

We consider the inversion problem for linear systems, which involves estimation of the unknown input vector. The inversion problem is considered for a system with a vector output and a vector input assuming that the observed output is of higher dimension than the unknown input. The problem is solved by using a controlled model in which the control stabilizes the deviations of the model output from the system output. The stabilizing model control or its averaged form may be used as the estimate of the unknown system input. __________ Translated from Nelineinaya Dinamika i Upravlenie, No. 4, pp. 17–22, 2004.     

New look-ahead Lanczos-type algorithms for linear systems

Summary. A breakdown (due to a division by zero) can arise in the algorithms for implementing Lanczos' method because of the non-existence of some formal orthogonal polynomials or because the recurrence relationship used is not appropriate. Such a breakdown can be avoided by jumping over the polynomials involved. This strategy was already used in some algorithms such as the MRZ and its variants. In this paper, we propose new implementations of the recurrence relations of these algorithms which only need the storage of a fixed number of vectors, independent of the length of the jump. These new algorithms are based on Horner's rule and on a different way for computing the coefficients of the recurrence relationships. Moreover, these new algorithms seem to be more stable than the old ones and they provide better numerical results. Numerical examples and comparisons with other algorithms will be given. Received September 2, 1997 / Revised version received July 24, 1998     

Ordering algorithms for irreducible sparse linear systems

Ordinary algorithms aim to pre-order a matrix in order to achieve a favourable structure for factorization. Two prototype structures are described which are commonly aimed at by current algorithms. A different structure, based on a lower Hessenberg form, is introduced. We show that the common structures may be obtained from the Hessenberg form in a simple way. A fundamental and very simple algorithm is presented for deriving the lower Hessenberg form. This algorithm is inherently a part of other common algorithms, but is usually obscured by other detailed heuristics. Some of these heuristics are seen to correspond to different tie-break rules in the fundamental algorithm. We describe a particularly simple tie-break rule used in SPK1 [11], which is effective and not at all well known. Ordinary algorithms need to be modified to enable pivoting for numerical stability to be carried out. We describe how the idea of threshold pivoting can be used in the context of these algorithms. Most ordering algorithms in common use employ some sort of look-ahead to promote a favourable structure. It is argued that look-ahead is generally ineffective in practice, and that numerical evidence supports the use of very simple strategies. A new ordering algorithm is presented which aims to obtain a narrower bandwidth in the lower Hessenberg form. Some limited numerical experience is presented which enables us to draw tentative conclusions about various ordering strategies, and how they compare with those in common use.     

Recursive blocked algorithms for linear systems with Kronecker product structure

Numerical Algorithms - Recursive blocked algorithms have proven to be highly efficient at the numerical solution of the Sylvester matrix equation and its generalizations. In this work, we show that...     

Benson type algorithms for linear vector optimization and applications

New versions and extensions of Benson’s outer approximation algorithm for solving linear vector optimization problems are presented. Primal and dual variants are provided in which only one scalar linear program has to be solved in each iteration rather than two or three as in previous versions. Extensions are given to problems with arbitrary pointed solid polyhedral ordering cones. Numerical examples are provided, one of them involving a new set-valued risk measure for multivariate positions.     

Optimally conditioned scaled ABS algorithms for linear systems

Using a strict bound of Spedicato to the condition number of bordered positive-definite matrices, we show that the scaling parameter in the ABS class for linear systems can always be chosen so that the bound of a certain update matrix is globally minimized. Moreover, if the scaling parameter is so chosen at every iteration, then the condition number itself is globally minimized. The resulting class of optimally conditioned algorithms contains as a special case the class of optimally stable algorithms in the sense of Broyden.This work was done in the framework of research supported by MPI, Rome, Italy, 60% Program.     

Some algorithms for inversion of linear dynamic systems

We study the problem of inversion of linear finite-dimensional dynamic systems in terms of the output. For different types of systems we propose several algorithms for inversion. We study the stability of the algorithms with respect to various perturbations. Translated fromAlgoritmy Upravleniya i Identifikatsii, pp. 156–167, 1997.