Parallel direct methods for solving the system of linear equations with pipelining on a multicore using OpenMP

Recent developments in high performance computer architecture have a significant effect on all fields of scientific computing. Linear algebra and especially the solution of linear systems of equations lie at the heart of many applications in scientific computing. This paper describes and analyzes three parallel versions of the dense direct methods such as the Gaussian elimination method and the LU form of Gaussian elimination that are used in linear system solving on a multicore using an OpenMP interface. More specifically, we present two naive parallel algorithms based on row block and row cyclic data distribution and we put special emphasis on presenting a third parallel algorithm based on the pipeline technique. Further, we propose an implementation of the pipelining technique in OpenMP. Experimental results on a multicore CPU show that the proposed OpenMP pipeline implementation achieves good overall performance compared to the other two naive parallel methods. Finally, in this work we propose a simple, fast and reasonably analytical model to predict the performance of the direct methods with the pipelining technique.     

Iterative methods for solving linear equations

This paper presents some of the original versions of the conjugate-gradient method for solving a system of linear equations of the formAx=k.This paper originally appeared as NAML Report No. 52-9, 1951. Its preparation was supported in part by the Office of Naval Research.     

Parallel iterative methods using factorized preconditioning matrices for solving elliptic equations on triangular grids

Parallel analogs of the variants of the incomplete Cholesky-conjugate gradient method and the modified incomplete Cholesky-conjugate gradient method for solving elliptic equations on uniform triangular and unstructured triangular grids on parallel computer systems with the MIMD architecture are considered. The construction of parallel methods is based on the use of various variants of ordering the grid points depending on the decomposition of the computation domain. Results of the theoretic and experimental studies of the convergence rate of these methods are presented. The solution of model problems on a moderate number processors is used to examine the efficiency of the proposed parallel methods.     

Certain approximate methods for solving linear operator equations

A solution of linear operator equations in the Hilbert space is constructed by using the best polynomial approximation of the inverse operator. This approach gives rise to certain iteration processes. Error estimates manifest that the suggested schemes may be fairly efficient.     

Numerical stability of descent methods for solving linear equations

Summary In this paper we perform a round-off error analysis of descent methods for solving a liner systemAx=b, whereA is supposed to be symmetric and positive definite. This leads to a general result on the attainable accuracy of the computed sequence {x _i } when the method is performed in floating point arithmetic. The general theory is applied to the Gauss-Southwell method and the gradient method. Both methods appear to be well-behaved which means that these methods compute an approximationx _i to the exact solutionA ^–1 b which is the exact solution of a slightly perturbed linear system, i.e. (A+A)x _i =b, A of order A, where is the relative machine precision and · denotes the spectral norm.     

Conjugate direction methods for solving systems of linear equations

A generalization of the notion of a set of directions conjugate to a matrix is shown to lead to a variety of finitely terminating iterations for solving systems of linear equations. The errors in the iterates are characterized in terms of projectors constructable from the conjugate directions. The natural relations of the algorithms to well known matrix decompositions are pointed out. Some of the algorithms can be used to solve linear least squares problems.This work was supported by the Office of Naval Research under contract number N 00014-67-A-0126.     

Two composition methods for solving certain systems of linear equations

Summary This note is concerned with the following problem: Given a systemA·x=b of linear equations and knowing that certains of its subsystemsA ¹·x ¹=b ¹, ...,A ^m ·x ^m =b ^m can be solved uniquely what can be said about the regularity ofA and how to find the solutionx fromx ¹, ...,x ^m ? This question is of particular interest for establishing methods computing certain linear or quasilinear sequence transformations recursively [7, 13, 15].Work performed under NATO Research Grant 027-81     

Characterization of linear stationary iterative processes for solving a singular system of linear equations

There are many iterative methods for solving a consistent singular system of linear equations [2, 3, 5, 6, 8, 10–13]. In this paper we characterize the class of all singular system of linear equations using generalized inverses.     

Parallel linear system solvers for Runge-Kutta methods

If the nonlinear systems arising in implicit Runge-Kutta methods like the Radau IIA methods are iterated by (modified) Newton, then we have to solve linear systems whose matrix of coefficients is of the form I-A hJ with A the Runge-Kutta matrix and J an approximation to the Jacobian of the righthand side function of the system of differential equations. For larger systems of differential equations, the solution of these linear systems by a direct linear solver is very costly, mainly because of the LU-decompositions. We try to reduce these costs by solving the linear systems by a second (inner) iteration process. This inner iteration process is such that each inner iteration again requires the solution of a linear system. However, the matrix of coefficients in these new linear systems is of the form I - B hJ where B is similar to a diagonal matrix with positive diagonal entries. Hence, after performing a similarity transformation, the linear systems are decoupled into s subsystems, so that the costs of the LU-decomposition are reduced to the costs of s LU-decompositions of dimension d. Since these LU-decompositions can be computed in parallel, the effective LU-costs on a parallel computer system are reduced by a factor s ³ . It will be shown that matrices B can be constructed such that the inner iterations converge whenever A and J have their eigenvalues in the positive and nonpositive halfplane, respectively. The theoretical results will be illustrated by a few numerical examples. A parallel implementation on the four-processor Cray-C98/4256 shows a speed-up ranging from at least 2.4 until at least 3.1 with respect to RADAU5 applied in one-processor mode.     

Quadrature-difference methods for solving linear and nonlinear singular integro-differential equations

Here we propose and justify quadrature-difference methods for solving different kinds (linear, nonlinear and multidimensional) of periodic singular integro-differential equations.     

General scheme for solving linear algebraic problems by direct methods

Assume that A is an m × n real matrix with m?n and rank(A)=n. Let b be a real vector with m components and consider the problem of finding $\text{x=A}{}^2\text{b}$, where $\text{A}{}^2\text{=(A}{}^{\mathrm{T}}\text{A)}{}^{\mathrm{?1}}\text{A}{}^{\mathrm{T}}$. A general scheme for solving this problem is described. Many well-known and commonly used in practice direct methods can be found as special cases within the general scheme. This is illustrated by four examples. The general scheme can be used to study the common properties of the direct methods. The usefulness of this approach in the efforts to improve the efficiency of the direct methods for sparse matrices is demonstrated by formulating an algorithm that can be applied to any particular method belonging to the general scheme. This algorithm is implemented in several subroutines solving linear algebraic problems by different direct methods. Numerical results, obtained in a wide range of runs with these subroutines, are given.     

Application of homotopy perturbation methods for solving systems of linear equations

In this paper, homotopy perturbation methods (HPMs) are applied to obtain the solution of linear systems, and conditions are deduced to check the convergence of the homotopy series. Moreover, we have adapted the Richardson method, the Jacobi method, and the Gauss-Seidel method to choose the splitting matrix. The numerical results indicate that the homotopy series converges much more rapidly than the direct methods for large sparse linear systems with a small spectrum radius.     

Quadrature methods of arbitrary order for solving linear ordinary differential equations

The writer shows that given any positive integern3 there is a one step method for numerical integration of the linear ordinary differential equationY=tAY+B of ordern+1, which employsn evaluations ofA andB. Numerical computations of the method whenn=3 and 4 compare quite favorably with the method of Runge-Kutta in those cases which are considered.     

Iterative refinement enhances the stability ofQR factorization methods for solving linear equations

Iterative refinement is a well-known technique for improving the quality of an approximate solution to a linear system. In the traditional usage residuals are computed in extended precision, but more recent work has shown that fixed precision is sufficient to yield benefits for stability. We extend existing results to show that fixed precision iterative refinement renders anarbitrary linear equations solver backward stable in a strong, componentwise sense, under suitable assumptions. Two particular applications involving theQR factorization are discussed in detail: solution of square linear systems and solution of least squares problems. In the former case we show that one step of iterative refinement suffices to produce a small componentwise relative backward error. Our results are weaker for the least squares problem, but again we find that iterative refinement improves a componentwise measure of backward stability. In particular, iterative refinement mitigates the effect of poor row scaling of the coefficient matrix, and so provides an alternative to the use of row interchanges in the HouseholderQR factorization. A further application of the results is described to fast methods for solving Vandermonde-like systems.     

On fast direct methods for solving elliptic equations over nonrectangular regions

In this paper, two methods based on the symmetric marching technique (SMT) are presented for the solution of elliptic equations over nonrectangular regions. Method I illustrates the direct adaptation of the SMT to irregular geometries. In method II, an efficient implementation of the capacitance matrix method has been considered using SMT. The favorable characteristics of the SMT for solving the Poisson equation with several right hand side functions and different boundary conditions without extra computational effort have been exploited for the last generation of the capacitance matrix. The successful application of the SMT combined with quasilinearization to solve mildly nonlinear elliptic equations is also described. Several test examples have been solved to demonstrate the efficiency of the proposed methods.     

On a modification of minimal iteration methods for solving systems of linear algebraic equations

Modifications of certain minimal iteration methods for solving systems of linear algebraic equations are proposed and examined. The modified methods are shown to be superior to the original versions with respect to the round-off error accumulation, which makes them applicable to solving ill-conditioned problems. Numerical results demonstrating the efficiency of the proposed modifications are given.     

Finite-difference methods for solving the reaction-diffusion equations of a simple isothermal chemical system

Numerical methods are proposed for the numerical solution of a system of reaction-diffusion equations, which model chemical wave propagation. The reaction terms in this system of partial differential equations contain nonlinear expressions. Nevertheless, it is seen that the numerical solution is obtained by solving a linear algebraic system at each time step, as opposed to solving a nonlinear algebraic system, which is often required when integrating nonlinear partial differential equations. The development of each numerical method is made in the light of experience gained in solving the system of ordinary differential equations, which model the well-stirred analogue of the chemical system. The first-order numerical methods proposed for the solution of this initialvalue problem are characterized to be implicit. However, in each case it is seen that the numerical solution is obtained explicitly. In a series of numerical experiments, in which the ordinary differential equations are solved first of all, it is seen that the proposed methods have superior stability properties to those of the well-known, first-order, Euler method to which they are compared. Incorporating the proposed methods into the numerical solution of the partial differential equations is seen to lead to two economical and reliable methods, one sequential and one parallel, for solving the travelling-wave problem. © 1994 John Wiley & Sons, Inc.