Numerical stability of descent methods for solving linear equations

Summary In this paper we perform a round-off error analysis of descent methods for solving a liner systemAx=b, whereA is supposed to be symmetric and positive definite. This leads to a general result on the attainable accuracy of the computed sequence {x _i } when the method is performed in floating point arithmetic. The general theory is applied to the Gauss-Southwell method and the gradient method. Both methods appear to be well-behaved which means that these methods compute an approximationx _i to the exact solutionA ^–1 b which is the exact solution of a slightly perturbed linear system, i.e. (A+A)x _i =b, A of order A, where is the relative machine precision and · denotes the spectral norm.     

Iterative refinement for constrained and weighted linear least squares

We present an algorithm for mixed precision iterative refinement on the constrained and weighted linear least squares problem, the CWLSQ problem. The approximate solution is obtained by solving the CWLSQ problem with the weightedQR factorization [6]. With backward errors for the weightedQR decomposition together with perturbation bounds for the CWLSQ problem we analyze the convergence behaviour of the iterative refinement procedure.In the unweighted case the initial convergence rate of the error of the iteratively refined solution is determined essentially by the condition number. For the CWLSQ problem the initial convergence behaviour is more complicated. The analysis shows that the initial convergence is dependent both on the condition of the problem related to the solution,x, and the vector =Wr, whereW is the weight matrix andr is the residual.We test our algorithm on two examples where the solution is known and the condition number of the problem can be varied. The computational test confirms the theoretical results and verifies that mixed precision iterative refinement, using the system matrix and the weightedQR decomposition, is an effective way of improving an approximate solution to the CWLSQ problem.     

Two composition methods for solving certain systems of linear equations

Summary This note is concerned with the following problem: Given a systemA·x=b of linear equations and knowing that certains of its subsystemsA ¹·x ¹=b ¹, ...,A ^m ·x ^m =b ^m can be solved uniquely what can be said about the regularity ofA and how to find the solutionx fromx ¹, ...,x ^m ? This question is of particular interest for establishing methods computing certain linear or quasilinear sequence transformations recursively [7, 13, 15].Work performed under NATO Research Grant 027-81     

Backward stability of a pivoting strategy for sign-regular linear systems

A matrixA issign-regular if, for each orderk, allk×k submatrices ofA have determinant with the same sign. In this paper, a pivoting strategy ofO(n) operations for the Gaussian elimination of linear systems whose coefficient matrices are sign-regular is proposed. Backward error analysis of this pivoting strategy is performed and small error bounds are obtained. Our results can also be applied to linear systems whose coefficient matrices have sign-regular inverses.     

Quadrature-difference methods for solving linear and nonlinear singular integro-differential equations

Here we propose and justify quadrature-difference methods for solving different kinds (linear, nonlinear and multidimensional) of periodic singular integro-differential equations.     

Preconditioned CG-type methods for solving the coupled system of fundamental semiconductor equations

This paper presents some of the authors' experimental results in applying Preconditioned CG-type methods to nonsymmetric systems of linear equations arising in the numerical solution of the coupled system of fundamental stationary semiconductor equations. For this type of problem it is shown that these iterative methods are efficient both in computation times and in storage requirements. All results have been obtained on an HP 350 computer.     

Error analysis of an algorithm for solving an underdetermined linear system

Summary The accumulation of rounding errors in a method used to compute the solution of an underdetermined system of linear equations at the least distance from a given point is being studied. The method used is based on orthogonal matrix decompositions.This research was partially supported by the Progetto Finalizzato Informatica of the Italian National Research Council     

A study of semiiterative methods for nonsymmetric systems of linear equations

Summary Given a nonsingular linear systemA x=b, a splittingA=M–N leads to the one-step iteration (1)x _m =T X _m–1 +c withT:=M ^–1N andc:=M ^–1 b. We investigate semiiterative methods (SIM's) with respect to (1), under the assumption that the eigenvalues ofT are contained in some compact set of , with 1. There exist SIM's which are optimal with respect to , but, except for some special sets , such optimal methods are not explicitly known in general. Using results about maximal convergence of polynomials and uniformly distributed nodes from approximation and function theory, we describe here SIM's which are asymptotically optimal with respect to . It is shown that Euler methods, extensively studied by Niethammer-Varga [NV], are special SIM's. Various algorithms for SIM's are also derived here. A 1-1 correspondence between Euler methods and SIM's, generated by generalized Faber polynomials, is further established here. This correspondence gives that asymptotically optimal Euler methods are quite near the optimal SIM's.Dedicated to Professor Karl Zeller (Universität Tübingen) on the occasion of his sixtieth birthday (December 28, 1984)     

Two-level hierarchically preconditioned conjugate gradient methods for solving linear elasticity finite element equations

In this paper we study and compare some preconditioned conjugate gradient methods for solving large-scale higher-order finite element schemes approximating two- and three-dimensional linear elasticity boundary value problems. The preconditioners discussed in this paper are derived from hierarchical splitting of the finite element space first proposed by O. Axelsson and I. Gustafsson. We especially focus our attention to the implicit construction of preconditioning operators by means of some fixpoint iteration process including multigrid techniques. Many numerical experiments confirm the efficiency of these preconditioners in comparison with classical direct methods most frequently used in practice up to now.     

Iterative methods for solving linear equations

This paper presents some of the original versions of the conjugate-gradient method for solving a system of linear equations of the formAx=k.This paper originally appeared as NAML Report No. 52-9, 1951. Its preparation was supported in part by the Office of Naval Research.     

Hybrid procedures for solving linear systems

Summary. In this paper, we introduce the notion of hybrid procedures for solving a system of linear equations. A hybrid procedure consists in a combination of two arbitrary approximate solutions with coefficients summing up to one. Thus the combination only depends on one parameter whose value is chosen in order to minimize the Euclidean norm of the residual vector obtained by the hybrid procedure. Properties of such procedures are studied in detail. The two approximate solutions which are combined in a hybrid procedure are usually obtained by two iterative methods. Several strategies for combining these two methods together or with the previous iterate of the hybrid procedure itself are discussed and their properties are analyzed. Numerical experiments illustrate the various procedures. Received October 21, 1992/Revised version received May 28, 1993     

Construction of Newton-like iteration methods for solving nonlinear equations

In this paper, we present a simple, and yet powerful and easily applicable scheme in constructing the Newton-like iteration formulae for the computation of the solutions of nonlinear equations. The new scheme is based on the homotopy analysis method applied to equations in general form equivalent to the nonlinear equations. It provides a tool to develop new Newton-like iteration methods or to improve the existing iteration methods which contains the well-known Newton iteration formula in logic; those all improve the Newton method. The orders of convergence and corresponding error equations of the obtained iteration formulae are derived analytically or with the help of Maple. Some numerical tests are given to support the theory developed in this paper.     

Perturbation results for nearly uncoupled Markov chains with applications to iterative methods

Summary The standard perturbation theory for linear equations states that nearly uncoupled Markov chains (NUMCs) are very sensitive to small changes in the elements. Indeed, some algorithms, such as standard Gaussian elimination, will obtain poor results for such problems. A structured perturbation theory is given that shows that NUMCs usually lead to well conditioned problems. It is shown that with appropriate stopping, criteria, iterative aggregation/disaggregation algorithms will achieve these structured error bounds. A variant of Gaussian elimination due to Grassman, Taksar and Heyman was recently shown by O'Cinneide to achieve such bounds.Supported by the National Science Foundation under grant CCR-9000526 and its renewal, grant CCR-9201692. This research was done in part, during the author's visit to the Institute for Mathematics and its Applications, 514 Vincent Hall, 206 Church St. S.E., University of Minnesota, Minneapolis, MN 55455, USA     

Three-step iterative methods with eighth-order convergence for solving nonlinear equations

A family of eighth-order iterative methods for the solution of nonlinear equations is presented. The new family of eighth-order methods is based on King’s fourth-order methods and the family of sixth-order iteration methods developed by Chun et al. Per iteration the new methods require three evaluations of the function and one evaluation of its first derivative. Therefore this family of methods has the efficiency index which equals 1.682. Kung and Traub conjectured that a multipoint iteration without memory based on n$n$ evaluations could achieve optimal convergence order 2ⁿ⁻¹$2^{n-1}$. Thus we provide a new example which agrees with the conjecture of Kung–Traub for n=4$n=4$. Numerical comparisons are made to show the performance of the presented methods.     

Block elimination with one refinement solves bordered linear systems accurately

Linear systems with a fairly well-conditioned matrixM of the form , for which a black box solver forA is available, can be accurately solved by the standard process of Block Elimination, followed by just one step of Iterative Refinement, no matter how singularA may be — provided the black box has a property that is possessed by LU- and QR-based solvers with very high probability. The resulting Algorithm BE + 1 is simpler and slightly faster than T.F. Chan's Deflation Method, and just as accurate. We analyse the case where the black box is a solver not forA but for a matrix close toA. This is of interest for numerical continuation methods.Dedicated to the memory of J. H. Wilkinson     

Component-wise perturbation analysis and error bounds for linear least squares solutions

Perturbation bounds for the linear least squares problem min _x Ax –b₂ corresponding tocomponent-wise perturbations in the data are derived. These bounds can be computed using a method of Hager and are often much better than the bounds derived from the standard perturbation analysis. In particular this is true for problems where the rows ofA are of widely different magnitudes. Generalizing a result by Oettli and Prager, we can use the bounds to compute a posteriori error bounds for computed least squares solutions.     

Solution of augmented linear systems using orthogonal factorizations

We consider solvingx+Ay=b andA ^T x=c for givenb, c andm ×n A of rankn. This is called the augmented system formulation (ASF) of two standard optimization problems, which include as special cases the minimum 2-norm of a linear underdetermined system (b=0) and the linear least squares problem (c=0), as well as more general problems. We examine the numerical stability of methods (for the ASF) based on the QR factorization ofA, whether by Householder transformations, Givens rotations, or the modified Gram-Schmidt (MGS) algorithm, and consider methods which useQ andR, or onlyR. We discuss the meaning of stability of algorithms for the ASF in terms of stability of algorithms for the underlying optimization problems.We prove the backward stability of several methods for the ASF which useQ andR, inclusing a new one based on MGS, and also show under what circumstances they may be regarded as strongly stable. We show why previous methods usingQ from MGS were not backward stable, but illustrate that some of these methods may be acceptable-error stable. We point out that the numerical accuracy of methods that do not useQ does not depend to any significant extent on which of of the above three QR factorizations is used. We then show that the standard methods which do not useQ are not backward stable or even acceptable-error stable for the general ASF problem, and discuss how iterative refinement can be used to counteract these deficiencies.Dedicated to Carl-Eric Fröberg on the occasion of his 75th birthdayThis research was partially supported by NSERC of Canada Grant No. A9236.     

Constrained least squares methods for linear timevarying DAE systems

Summary This paper presents a family of methods for accurate solution of higher index linear variable DAE systems, . These methods use the DAE system and some of its first derivatives as constraints to a least squares problem that corresponds to a Taylor series ofy, or an approximative equality derived from a Pade' approximation of the exponential function. Accuracy results for systems transformable to standard canonical form are given. Advantages, disadvantages, stability properties and implementation of these methods are discussed and two numerical examples are given, where we compare our results with results from more traditional methods.     

Block-iterative methods for consistent and inconsistent linear equations

Summary We shall in this paper consider the problem of computing a generalized solution of a given linear system of equations. The matrix will be partitioned by blocks of rows or blocks of columns. The generalized inverses of the blocks are then used as data to Jacobi- and SOR-types of iterative schemes. It is shown that the methods based on partitioning by rows converge towards the minimum norm solution of a consistent linear system. The column methods converge towards a least squares solution of a given system. For the case with two blocks explicit expressions for the optimal values of the iteration parameters are obtained. Finally an application is given to the linear system that arises from reconstruction of a two-dimensional object by its one-dimensional projections.     

On structure-oriented hybrid two-stage iteration methods for the large and sparse blocked system of linear equations

In this paper, we first present a class of structure-oriented hybrid two-stage iteration methods for solving the large and sparse blocked system of linear equations, as well as the saddle point problem as a special case. And the new methods converge to the solution under suitable restrictions, for instance, when the coefficient matrix is positive stable matrix generally. Numerical experiments for a model generalized saddle point problem are given, and the results show that our new methods are feasible and efficient, and converge faster than the Classical Uzawa Method.