Parallel nonlinear multisplitting methods

Summary Linear multisplitting methods are known as parallel iterative methods for solving a linear systemAx=b. We extend the idea of multisplittings to the problem of solving a nonlinear system of equationsF(x)=0. Our nonlinear multisplittings are based on several nonlinear splittings of the functionF. In a parallel computing environment, each processor would have to calculate the exact solution of an individual nonlinear system belonging to his nonlinear multisplitting and these solutions are combined to yield the next iterate. Although the individual systems are usually much less involved than the original system, the exact solutions will in general not be available. Therefore, we consider important variants where the exact solutions of the individual systems are approximated by some standard method such as Newton's method. Several methods proposed in literature may be regarded as special nonlinear multisplitting methods. As an application of our systematic approach we present a local convergence analysis of the nonlinear multisplitting methods and their variants. One result is that the local convergence of these methods is determined by an induced linear multisplitting of the Jacobian ofF.Dedicated to the memory of Peter Henrici     

Parallel QR decomposition of a rectangular matrix

Summary We show that the greedy algorithm introduced in [1] and [5] to perform the parallel QR decomposition of a dense rectangular matrix of sizem×n is optimal. Then we assume thatm/n ² tends to zero asm andn go to infinity, and prove that the complexity of such a decomposition is asymptotically2n, when an unlimited number of processors is available.     

On a theorem of Stein-Rosenberg type in interval analysis

Summary In classical numerical analysis the asymptotic convergence factor (R ₁-factor) of an iterative processx ^m+1=Ax^m+b coincides with the spectral radius of then×n iteration matrixA. Thus the famous Theorem of Stein and Rosenberg can at least be partly reformulated in terms of asymptotic convergence factor. Forn×n interval matricesA with irreducible upper bound and nonnegative lower bound we compare the asymptotic convergence factor ( _T ) of the total step method in interval analysis with the factor _S of the corresponding single step method. We derive a result similar to that of the Theorem of Stein and Rosenberg. Furthermore we show that _S can be less than the spectral radius of the real single step matrix corresponding to the total step matrix |A| where |A| is the absolute value ofA. This answers an old question in interval analysis.     

On the convergence of powers of interval matrices (2)

Summary Let be a real irreduciblen×n interval matrix. Then a necessary and sufficient condition is given for the sequence of the powers of an interval matrix to converge to a matrix which is not the null matrix. In addition a criterion for is proved to decide whether the limit matrix satisfies the condition of symmetry .     

Convergence theory of extrapolated iterative methods for a certain class of non-symmetric linear systems

Summary A variety of iterative methods considered in [3] are applied to linear algebraic systems of the formAu=b, where the matrixA is consistently ordered [12] and the iteration matrix of the Jacobi method is skew-symmetric. The related theory of convergence is developed and the optimum values of the involved parameters for each considered scheme are determined. It reveals that under the aforementioned assumptions the Extrapolated Successive Underrelaxation method attains a rate of convergence which is clearly superior over the Successive Underrelaxation method [5] when the Jacobi iteration matrix is non-singular.     

The rate of convergence of Conjugate Gradients

Summary It has been observed that the rate of convergence of Conjugate Gradients increases when one or more of the extreme Ritz values have sufficiently converged to the corresponding eigenvalues (the superlinear convergence of CG). In this paper this will be proved and made quantitative. It will be shown that a very modest degree of convergence of an extreme Ritz value already suffices for an increased rate of convergence to occur.     

Optimum stationary and nonstationary iterative methods for the solution of singular linear systems

Summary A number of iterative methods for the solution of the singular linear systemAx=b (det(A)=0 andb in the range ofA) is analyzed and studied. Among them are the Stationaryk-Step, the Accelerated Overrelaxation (AOR) and the Nonstationary Second Order Chebyshev Semi-Iterative ones. It is proved that, under certain assumptions, the corresponding optimum semiconvergent schemes, which present a great resemblance with their analogs for the nonsingular case, can be determined. Finally, a number of numerical examples shows how one can use the theory to obtain the optimum parameters for each applicable semiconvergent method.     

A note on comparison theorems for nonnegative matrices

Summary In a recent paper, [4], Csordas and Varga have unified and extended earlier theorems, of Varga in [10] and Wonicki in [11], on the comparison of the asymptotic rates of convergence of two iteration matrices induced by two regular splittings. The main purpose of this note is to show a connection between the Csordas-Varga paper and a paper by Beauwens, [1], in which a comparison theorem is developed for the asymptotic rate of convergence of two nonnegative iteration matrices induced by two splittings which are not necessarily regular. Monotonic norms already used in [1] play an important role in our work here.Research supported in part by NSF grant number DMS-8400879     

Fast inversion algorithms of Toeplitz-plus-Hankel matrices

Summary The paper deals with the problems of fast inversion of matricesA=T+H, whereT is Toeplitz andH is Hankel. Several algorithms are presented and compared, among them algorithms working for arbitrary strongly nonsingular matricesA=T+H.     

QR factorization of Toeplitz matrices

Summary This paper presents a new algorithm for computing theQR factorization of anm×n Toeplitz matrix inO(mn) operations. The algorithm exploits the procedure for the rank-1 modification and the fact that both principal (m–1)×(n–1) submatrices of the Toeplitz matrix are identical. An efficient parallel implementation of the algorithm is possible.     

The local convergence of ABS methods for nonlinear algebraic equations

Summary In this paper we consider an extension to nonlinear algebraic systems of the class of algorithms recently proposed by Abaffy, Broyden and Spedicato for general linear systems. We analyze the convergence properties, showing that under the usual assumptions on the function and some mild assumptions on the free parameters available in the class, the algorithm is locally convergent and has a superlinear rate of convergence (per major iteration, which is computationally comparable to a single Newton's step). Some particular algorithms satisfying the conditions on the free parameters are considered.     

Error analysis of the Björck-Pereyra algorithms for solving Vandermonde systems

Summary A forward error analysis is presented for the Björck-Pereyra algorithms used for solving Vandermonde systems of equations. This analysis applies to the case where the points defining the Vandermonde matrix are nonnegative and are arranged in increasing order. It is shown that for a particular class of Vandermonde problems the error bound obtained depends on the dimensionn and on the machine precision only, being independent of the condition number of the coefficient matrix. By comparing appropriate condition numbers for the Vandermonde problem with the forward error bounds it is shown that the Björck-Pereyra algorithms introduce no more uncertainty into the numerical solution than is caused simply by storing the right-hand side vector on the computer. A technique for computing running a posteriori error bounds is derived. Several numerical experiments are presented, and it is observed that the ordering of the points can greatly affect the solution accuracy.     

Tchebychev acceleration technique for large scale nonsymmetric matrices

Summary The acceleration by Tchebychev iteration for solving nonsymmetric eigenvalue problems is dicussed. A simple algorithm is derived to obtain the optimal ellipse which passes through two eigenvalues in a complex plane relative to a reference complex eigenvalue. New criteria are established to identify the optimal ellipse of the eigenspectrum. The algorithm is fast, reliable and does not require a search for all possible ellipses which enclose the spectrum. The procedure is applicable to nonsymmetric linear systems as well.     

Optimality relationships forp-cyclic SOR

Summary The optimality question for blockp-cyclic SOR iterations discussed in Young and Varga is answered under natural conditions on the spectrum of the block Jacobi matrix. In particular, it is shown that repartitioning a blockp-cyclic matrix into a blockq-cyclic form,q, results in asymptotically faster SOR convergence for the same amount of work per iteration. As a consequence block 2-cyclic SOR is optimal under these conditions.Research supported in part by the US Air Force under Grant no. AFOSR-88-0285 and the National Science Foundation under grant no. DMS-85-21154 Present address: Boeing Computer Services, P.O. Box 24346, MS 7L-21, Seattle, WA 98124-0346, USA     

A generalized conjugate gradient,least square method

Summary A generalizeds-term truncated conjugate gradient method of least square type, proposed in [1a, b], is extended to a form more suitable for proving when the truncated version is identical to the full-term version. Advantages with keeping a control term in the truncated version is pointed out. A computationally efficient new algorithm, based on a special inner product with a small demand of storage is also presented.We also give simplified and slightly extended proofs of termination of the iterative sequence and of existence of ans-term recursion, identical to the full-term version. Important earlier results on this latter topic are found in [15, 16, 8 and 11].The research reported in this paper was partly supported by NATO Grant No. 648/83     

On the monotonity of incomplete factorization

Summary The Meijerink, van der Vorst type incomplete decomposition uses a position set, where the factors must be zero, but their product may differ from the original matrix. The smaller this position set is, the more the product of incomplete factors resembles the original matrix. The aim of this paper is to discuss this type of monotonity. It is shown using the Perron Frobenius theory of nonnegative matrices, that the spectral radius of the iteration matrix is a monotone function of the position set. On the other hand no matrix norm of the iteration matrix depends monotonically on the position set. Comparison is made with the modified incomplete factorization technique.     

Extrapolated Gauss-Seidel I and SOR methods for least-squares problems

Summary Recently, special attention has been given in the literature, to the problems of accurately computing the least-squares solution of very largescale over-determined systems of linear equations which occur in geodetic applications. In particular, it has been suggested that one can solve such problems iteratively by applying the block SOR (Successive Overrelaxation) and EGS1 (Extrapolated Gauss Seidel 1) plus semi-iterative methods to a linear system with coefficient matrix 2-cyclic or 3-cyclic. The comparison of 2-block SOR and 3-block SOR was made in [1] and showed that the 2-block SOR is better. In [6], the authors also proved that 3-block EGS1-SI is better than 3-block SOR. Here, we first show that the 2-block DJ (Double Jacobi)-SI, GS-SI and EGS1-SI methods are equivalent and all of them are equivalent to the 3-block EGS1-SI method; then, we prove that the composite methods and 2-block SOR have the same asymptotic rate of convergence, but the former has a better average rate of convergence; finally, numerical experiments are reported, and confirm that the 3-block EGS1-SI is better than the 2-block SOR.     

The convergence of inexact Chebyshev and Richardson iterative methods for solving linear systems

Summary The Chebyshev and second-order Richardson methods are classical iterative schemes for solving linear systems. We consider the convergence analysis of these methods when each step of the iteration is carried out inexactly. This has many applications, since a preconditioned iteration requires, at each step, the solution of a linear system which may be solved inexactly using an inner iteration. We derive an error bound which applies to the general nonsymmetric inexact Chebyshev iteration. We show how this simplifies slightly in the case of a symmetric or skew-symmetric iteration, and we consider both the cases of underestimating and overestimating the spectrum. We show that in the symmetric case, it is actually advantageous to underestimate the spectrum when the spectral radius and the degree of inexactness are both large. This is not true in the case of the skew-symmetric iteration. We show how similar results apply to the Richardson iteration. Finally, we describe numerical experiments which illustrate the results and suggest that the Chebyshev and Richardson methods, with reasonable parameter choices, may be more effective than the conjugate gradient method in the presence of inexactness.This work was supported in part by National Science Foundation Grants DCR-8412314 and DCR-8502014The work of this author was completed while he was on sabbatical leave at the Centre for Mathematical Analysis and Mathematical Sciences Research Institute at the Australian National University, Canberra, Australia     

Generalizations of the projection method with applications to SOR theory for hermitian positive semidefinite linear systems

Summary Ann×n complex matrixB is calledparacontracting if B₂1 and 0x[N(I-B)]Bx₂<x₂. We show that a productB=B _k B _k–1 ...B ₁ ofk paracontracting matrices is semiconvergent and give upper bounds on the subdominant eigenvalue ofB in terms of the subdominant singular values of theB _i 's and in terms of the angles between certain subspaces. Our results here extend earlier results due to Halperin and due to Smith, Solomon and Wagner. We also determine necessary and sufficient conditions forn numbers in the interval [0, 1] to form the spectrum of a product of two orthogonal projections and hence characterize the subdominant eigenvalue of such a product. In the final part of the paper we apply the upper bounds mentioned earlier to provide an estimate on the subdominant eigenvalue of the SOR iteration matrix associated with ann×n hermitian positive semidefinite matrixA none of whose diagonal entries vanish.The work of this author was supported in part by NSF Research Grant No. MCS-8400879     

Optimal successive overrelaxation iterative methods for P-cyclic matrices

Summary We consider linear systems whose associated block Jacobi matricesJ _p are weakly cyclic of indexp. In a recent paper, Pierce, Hadjidimos and Plemmons [13] proved that the block two-cyclic successive overrelaxation (SOR) iterative method is numerically more effective than the blockq-cyclic SOR-method, 2<qp, if the eigenvalues ofJ _p ^p are either all non-negative or all non-positive. Based on the theory of stationaryp-step methods, we give an alternative proof of their theorem. We further determine the optimal relaxation parameter of thep-cyclic SOR method under the assumption that the eigenvalues ofJ _p ^p are contained in a real interval, thereby extending results due to Young [19] (for the casep=2) and Varga [15] (forp>2). Finally, as a counterpart to the result of Pierce, Hadjidimos and Plemmons, we show that, under this more general assumption, the two-cyclic SOR method is not always superior to theq-cyclic SOR method, 2<qp.Dedicated to R. S. Varga on the occasion of his 60th birthdayResearch supported in part by the Deutsche Forschungsgemeinschaft