Growth factors of pivoting strategies associated with Neville elimination

Neville elimination is a direct method for solving linear systems. Several pivoting strategies for Neville elimination, including pairwise pivoting, are analyzed. Bounds for two different kinds of growth factors are provided. Finally, an approximation of the average normalized growth factor associated with several pivoting strategies is computed and analyzed using random matrices.     

Growth factor and expected growth factor of some pivoting strategies

Several definitions of growth factors for Gaussian elimination are compared. Some new pivoting strategies, intermediate between partial pivoting and rook pivoting, are introduced. For random matrices, an approximation of the average normalized growth factor associated with several pivoting strategies is computed and analyzed. A stationary behavior of the expected growth factors of the new pivoting strategies is observed. Bounds for the growth factors of these pivoting strategies are provided. It is also shown that partial pivoting by columns produces small growth factors for matrices appearing in practical observations and for which the growth factors produced by partial pivoting are very large.     

Sign regular matrices and Neville elimination

A pivoting strategy of O(n) operations for the Neville elimination of n × n nonsingular sign regular matrices is introduced. Among other nice properties, it is proved that it preserves sign regularity. It is also shown its relationship with scaled partial pivoting strategies for Neville elimination.     

Pivoting strategies leading to diagonal dominance by rows

Pivoting strategies for Gaussian elimination leading to upper triangular matrices which are diagonally dominant by rows are studied. Forward error analysis of triangular systems whose coefficient matrices are diagonally dominant by rows is performed. We also obtain small bounds of the backward errors for the pivoting strategies mentioned above. Our examples of matrices include H-matrices and some generalizations of diagonally dominant matrices, and scaled partial pivoting for the 1-norm is an example of these pivoting strategies. In the case of an M-matrix, a pivoting strategy of computational complexity is proposed, which satisfies all the results of the paper. Received June 6, 1997 / Revised version received October 27, 1997     

Backward stability of a pivoting strategy for sign-regular linear systems

A matrixA issign-regular if, for each orderk, allk×k submatrices ofA have determinant with the same sign. In this paper, a pivoting strategy ofO(n) operations for the Gaussian elimination of linear systems whose coefficient matrices are sign-regular is proposed. Backward error analysis of this pivoting strategy is performed and small error bounds are obtained. Our results can also be applied to linear systems whose coefficient matrices have sign-regular inverses.     

Stable solution of tridiagonal systems

In this paper we present three different pivoting strategies for solving general tridiagonal systems of linear equations. The first strategy resembles the classical method of Gaussian elimination with no pivoting and is stable provided a simple and easily checkable condition is met. In the second strategy, the growth of the elements is monitored so as to ensure backward stability in most cases. Finally, the third strategy also uses the right‐hand side vector to make pivoting decisions and is proved to be unconditionally backward stable. This revised version was published online in August 2006 with corrections to the Cover Date.     

Symmetric Gaussian elimination for cauchy-type matrices with application to positive definite Toeplitz matrices

The problem of solving linear equations with a Toeplitz matrix appears in many applications. Often is positive definite but ill-conditioned with many small eigenvalues. In this case fast and superfast algorithms may show a very poor behavior or even break down. In recent papers the transformation of a Toeplitz matrix into a Cauchy-type matrix is proposed. The resulting new linear equations can be solved in operations using standard pivoting strategies which leads to very stable fast methods also for ill-conditioned systems. The basic tool is the formulation of Gaussian elimination for matrices with low displacement rank. In this paper, we will transform a Hermitian Toeplitz matrix into a Cauchy-type matrix by applying the Fourier transform. We will prove some useful properties of and formulate a symmetric Gaussian elimination algorithm for positive definite . Using the symmetry and persymmetry of we can reduce the total costs of this algorithm compared with unsymmetric Gaussian elimination. For complex Hermitian , the complexity of the new algorithm is then nearly the same as for the Schur algorithm. Furthermore, it is possible to include some strategies for ill-conditioned positive definite matrices that are well-known in optimization. Numerical examples show that this new algorithm is fast and reliable. Received March 24, 1995 / Revised version received December 13, 1995     

D-optimal designs embedded in Hadamard matrices and their effect on the pivot patterns

In this paper we develop a new approach for detecting if specific D-optimal designs exist embedded in Sylvester-Hadamard matrices. Specifically, we investigate the existence of the D-optimal designs of orders 5, 6, 7 and 8. The problem is motivated to explaining why specific values appear as pivot elements when Gaussian elimination with complete pivoting is applied to Hadamard matrices. Using this method and a complete search algorithm we explain, for the first time, the appearance of concrete pivot values for equivalence classes of Hadamard matrices of orders n = 12, 16 and 20.     

A collection of examples where Neville elimination outperforms Gaussian elimination

Neville elimination is an elimination procedure alternative to Gaussian elimination. It is very useful when dealing with totally positive matrices, for which nice stability results are known. Here we include examples, most of them test matrices used in MATLAB which are not totally positive matrices, where Neville elimination outperforms Gaussian elimination.     

Pivoting strategy for rank-one modification ofLDM t-like factorization

The rank-one modification algorithm of theLDM ^t factorization was given by Bennett [1]. His method, however, could break down even when the matrix is nonsingular and well-conditioned. We introduce a pivoting strategy for avoiding possible break-down as well as for suppressing error growth in the modification process. The method is based on a symbolic formula of the rank-one modification of the factorization of a possibly singular nonsymmetric matrix. A new symbolic formula is also obtained for the inverses of the factor matrices. Repeated application of our method produces theLDM ^t-like product form factorization of a matrix. A numerical example is given to illustrate our pivoting method. An incomplete factorization algorithm is also introduced for updating positive definite matrix useful in quasi-Newton methods, in which the Fletcher and Powell algorithm [2] and the Gill, Murray and Saunders algorithm [4] are usually used.This paper is presented at the Japan SIAM Annual Meeting held at University of Tokyo, Japan, October 7–9, 1991.     

Neville elimination for rank-structured matrices

In this paper it is shown that Neville elimination is suited to exploit the rank structure of an order-r quasiseparable matrix A∈C^n×n by providing a condensed decomposition of A as product of unit bidiagonal matrices, all together specified by O(nr) parameters, at the cost of O(nr³) flops. An application of this result for eigenvalue computation of totally positive rank-structured matrices is also presented.     

On a direct method for the solution of nearly uncoupled Markov chains

Summary This note is concerned with the accuracy of the solution of nearly uncoupled Markov chains by a direct method based on the LU decomposition. It is shown that plain Gaussian elimination may fail in the presence of rounding errors. A modification of Gaussian elimination with diagonal pivoting and correction of small pivots is proposed and analyzed. It is shown that the accuracy of the solution is affected by two condition numbers associated with aggregation and the coupling respectively.This work was supported in part by the Air Force Office of Sponsored Research under Contract AFOSR-87-0188     

Some New Developments in ILU Preconditioners

ILUC, a Crout-based Gaussian elimination incomplete LDU-factorization preconditioner, can be improved for some matrices by pivoting by columns and by interchanging rows for better sparsity. Just as many other ILU-type preconditioners, this can implemented in a multilevel framework and combined with various dropping strategies. (© 2006 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

A backward stability analysis of diagonal pivoting methods for solving unsymmetric tridiagonal systems without interchanges

This paper concerns the LBM ^T factorization of unsymmetric tridiagonal matrices, where L and M are unit lower triangular matrices and B is block diagonal with 1×1 and 2×2 blocks. In some applications, it is necessary to form this factorization without row or column interchanges while the tridiagonal matrix is formed. Bunch and Kaufman proposed a pivoting strategy without interchanges specifically for symmetric tridiagonal matrices, and more recently, Bunch and Marcia proposed pivoting strategies that are normwise backward stable for linear systems involving such matrices. In this paper, we extend these strategies to the unsymmetric tridiagonal case and demonstrate that the proposed methods both exhibit bounded growth factors and are normwise backward stable. Copyright © 2009 John Wiley & Sons, Ltd.     

Time-stepping and preserving orthonormality

Certain applications produce initial value ODEs whose solutions, regarded as time-dependent matrices, preserve orthonormality. Such systems arise in the computation of Lyapunov exponents and the construction of smooth singular value decompositions of parametrized matrices. For some special problem classes, there exist time-stepping methods that automatically inherit the orthonormality preservation. However, a more widely applicable approach is to apply a standard integrator and regularly replace the approximate solution by an orthonormal matrix. Typically, the approximate solution is replaced by the factorQ from its QR decomposition (computed, for example, by the modified Gram-Schmidt method). However, the optimal replacement—the one that is closest in the Frobenius norm—is given by the orthonormal polar factor. Quadratically convergent iteration schemes can be used to compute this factor. In particular, there is a matrix multiplication based iteration that is ideally suited to modern computer architectures. Hence, we argue that perturbing towards the orthonormal polar factor is an attractive choice, and we consider performing a fixed number of iterations. Using the optimality property we show that the perturbations improve the departure from orthonormality without significantly degrading the finite-time global error bound for the ODE solution. Our analysis allows for adaptive time-stepping, where a local error control process is driven by a user-supplied tolerance. Finally, using a recent result of Sun, we show how the global error bound carries through to the case where the orthonormal QR factor is used instead of the orthonormal polar factor. This work was supported by Engineering and Physical Sciences Research Council grants GR/H94634 and GR/K80228.     

Simultaneous backward stability of Gauss and Gauss–Jordan elimination

It is well known that some pivoting strategies are backward stable for Gauss elimination but are not backward stable for Gauss–Jordan elimination (GJE) when these procedures are used to solve a linear system Ax=b. We analyse the simultaneous backward stability for Gauss and GJE of several pivoting strategies, including a pivoting strategy which we call double partial pivoting. Copyright © 2002 John Wiley & Sons, Ltd.     

Partial pivoting strategies for symmetric gaussian elimination

In an earlier paper [6] it is shown how the use of symmetric additions of rows and columns enables a stableLDL ^T factorization of symmetric indefinite matrices. In this paper we describe partial pivoting strategies. These strategies are faster than the complete pivoting strategies that were introduced in the first paper. Numerical experiments are included. The results show that some of the new strategies share the stable behaviour of complete pivoting while running almost as fast as the well-known Cholesky decomposition.     

Matrix computations and polynomial root-finding with preprocessing

Solution of homogeneous linear systems of equations is a basic operation of matrix computations. The customary algorithms rely on pivoting, orthogonalization and SVD, but we employ randomized preprocessing instead. This enables us to accelerate the solution dramatically, both in terms of the estimated arithmetic cost and the observed CPU time. The approach is effective in the cases of both general and structured input matrices and we extend it and its computational advantages to the solution of nonhomogeneous linear systems of equations, matrix eigen-solving, the solution of polynomial and secular equations, and approximation of a matrix by a nearby matrix that has a smaller rank or a fixed structure (e.g., of the Toeplitz or Hankel type). Our analysis and extensive experiments show the power of the presented algorithms.     

Remarks on a unified theory for classical and generalized interpolation and extrapolation

A unified theory for generalized interpolation, as developed by Mühlbach, and classical polynomial interpolation is discussed. A fundamental theorem for generalized linear iterative interpolation is given and used to derive generalizations of the classical formulae due to Neville, Aitken and Lagrange. Using Mühlbach's definition of generalized divided differences, Newton's generalized interpolation formula, including an expression for the error term, is derived as a pure identity.