Convergence of CG and GMRES on a tridiagonal Toeplitz linear system

The Conjugate Gradient method (CG), the Minimal Residual method (MINRES), or more generally, the Generalized Minimal Residual method (GMRES) are widely used to solve a linear system Ax=b. The choice of a method depends on A’s symmetry property and/or definiteness), and MINRES is really just a special case of GMRES. This paper establishes error bounds on and sometimes exact expressions for residuals of CG, MINRES, and GMRES on solving a tridiagonal Toeplitz linear system, where A is Hermitian or just normal. These expressions and bounds are in terms of the three parameters that define A and Chebyshev polynomials of the first or second kind. AMS subject classification (2000)  65F10, 65N22     

GMRES and the Arioli,Pták,and Strakoš parametrization

In this paper we study the parametrization proposed by Arioli, Pták and Strakoš (BIT Numerical Mathematics, v 38, 1998) for the class of matrices having the same GMRES residual norm convergence curve. We give expressions for the Hessenberg matrix and the orthonormal basis vectors constructed by GMRES as well as for iterates and error vectors. The iterates do not depend on the eigenvalues in the sense that changing the coefficients of the characteristic polynomial in the parametrization does not change the GMRES iterates as well as the residuals. However, the error vectors do depend on these coefficients.     

Iterative methods for solvingAx=b,GMRES/FOM versus QMR/BiCG

We study the convergence of the GMRES/FOM and QMR/BiCG methods for solving nonsymmetric systems of equationsAx=b. We prove, in exact arithmetic, that any type of residual norm convergence obtained using BiCG can also be obtained using FOM but on a different system of equations. We consider practical comparisons of these procedures when they are applied to the same matrices. We use a unitary invariance shared by both methods, to construct test matrices where we can vary the nonnormality of the test matrix by variations in simplified eigenvector matrices. We used these test problems in two sets of numerical experiments. The first set of experiments was designed to study effects of increasing nonnormality on the convergence of GMRES and QMR. The second set of experiments was designed to track effects of the eigenvalue distribution on the convergence of QMR. In these tests the GMRES residual norms decreased significantly more rapidly than the QMR residual norms but without corresponding decreases in the error norms. Furthermore, as the nonnormality ofA was increased, the GMRES residual norms decreased more rapidly. This led to premature termination of the GMRES procedure on highly nonnormal problems. On the nonnormal test problems the QMR residual norms exhibited less sensitivity to changes in the nonnormality. The convergence of either type of procedure, as measured by the error norms, was delayed by the presence of large or small outliers and affected by the type of eigenvalues, real or complex, in the eigenvalue distribution ofA. For GMRES this effect can be seen only in the error norm plots.In honor of the 70th birthday of Ted RivlinThis work was supported by NSF grant GER-9450081.     

On the convergence behavior of the restarted GMRES algorithm for solving nonsymmetric linear systems

The solution of nonsymmetric systems of linear equations continues to be a difficult problem. A main algorithm for solving nonsymmetric problems is restarted GMRES. The algorithm is based on restarting full GMRES every s iterations, for some integer s>0. This paper considers the impact of the restart frequency s on the convergence and work requirements of the method. It is shown that a good choice of this parameter can lead to reduced solution time, while an improper choice may hinder or preclude convergence. An adaptive procedure is also presented for determining automatically when to restart. The results of numerical experiments are presented.     

GMRES and the minimal polynomial

We present a qualitative model for the convergence behaviour of the Generalised Minimal Residual (GMRES) method for solving nonsingular systems of linear equationsAx =b in finite and infinite dimensional spaces. One application of our methods is the solution of discretised infinite dimensional problems, such as integral equations, where the constants in the asymptotic bounds are independent of the mesh size.Our model provides simple, general bounds that explain the convergence of GMRES as follows: If the eigenvalues ofA consist of a single cluster plus outliers then the convergence factor is bounded by the cluster radius, while the asymptotic error constant reflects the non-normality ofA and the distance of the outliers from the cluster. If the eigenvalues ofA consist of several close clusters, then GMRES treats the clusters as a single big cluster, and the convergence factor is the radius of this big cluster. We exhibit matrices for which these bounds are tight.Our bounds also lead to a simpler proof of existing r-superlinear convergence results in Hilbert space.This research was partially supported by National Science Foundation grants DMS-9122745, DMS-9423705, CCR-9102853, CCR-9400921, DMS-9321938, DMS-9020915, and DMS-9403224.     

Consistent estimator in multivariate errors-in-variables model in the case of unknown error covariance structure

We consider a linear multivariate errors-in-variables model AX ≈ B, where the matrices A and B are observed with errors and the matrix parameter X is to be estimated. In the case of lack of information about the error covariance structure, we propose an estimator that converges in probability to X as the number of rows in A tends to infinity. Sufficient conditions for this convergence and for the asymptotic normality of the estimator are found. __________ Translated from Ukrains’kyi Matematychnyi Zhurnal, Vol. 59, No. 8, pp. 1026–1033, August, 2007.     

Convergence conditions for a restarted GMRES method augmented with eigenspaces

We consider the GMRES(m,k) method for the solution of linear systems Ax=b, i.e. the restarted GMRES with restart m where to the standard Krylov subspace of dimension m the other subspace of dimension k is added, resulting in an augmented Krylov subspace. This additional subspace approximates usually an A‐invariant subspace. The eigenspaces associated with the eigenvalues closest to zero are commonly used, as those are thought to hinder convergence the most. The behaviour of residual bounds is described for various situations which can arise during the GMRES(m,k) process. The obtained estimates for the norm of the residual vector suggest sufficient conditions for convergence of GMRES(m,k) and illustrate that these augmentation techniques can remove stagnation of GMRES(m) in many cases. All estimates are independent of the choice of an initial approximation. Conclusions and remarks assessing numerically the quality of proposed bounds conclude the paper. Copyright © 2004 John Wiley & Sons, Ltd.     

The rate of convergence of GMRES on a tridiagonal toeplitz linear system. II

This paper continues the recent work of the authors’ [R.-C. Li, W. Zhang, The rate of convergence of GMRES on a tridiagonal Toeplitz linear system, Numer. Math. 112 (2009) 267-293 (electronically published on 19 December 2008)] on the rate of convergence of GMRES for a tridiagonal Toeplitz linear system Ax=b. Much simpler formulas than the earlier ones for GMRES residuals when b is the first or the last column of the identity matrix are established, and these formulas allow us to confirm the rate of convergence that was conjectured but only partially proven earlier. Simpler and sharper bounds than earlier ones when all b’s entries, except its first and last ones, are zeros are also obtained.     

A geometric analysis of Renegar’s condition number,and its interplay with conic curvature

For a conic linear system of the form Ax ∈K, K a convex cone, several condition measures have been extensively studied in the last dozen years. Among these, Renegar’s condition number is arguably the most prominent for its relation to data perturbation, error bounds, problem geometry, and computational complexity of algorithms. Nonetheless, is a representation-dependent measure which is usually difficult to interpret and may lead to overly conservative bounds of computational complexity and/or geometric quantities associated with the set of feasible solutions. Herein we show that Renegar’s condition number is bounded from above and below by certain purely geometric quantities associated with A and K; furthermore our bounds highlight the role of the singular values of A and their relationship with the condition number. Moreover, by using the notion of conic curvature, we show how Renegar’s condition number can be used to provide both lower and upper bounds on the width of the set of feasible solutions. This complements the literature where only lower bounds have heretofore been developed.     

The convergence of Krylov subspace methods for large unsymmetric linear systems

The convergence problem of many Krylov subspace methods,e.g., FOM, GCR, GMRES and QMR, for solving large unsymmetric (non-Hermitian) linear systems is considered in a unified way when the coefficient matrixA is defective and its spectrum lies in the open right (left) half plane. Related theoretical error bounds are established and some intrinsic relationships between the convergence speed and the spectrum ofA are exposed. It is shown that these methods are likely to converge slowly once one of the three cases occurs:A is defective, the distribution of its spectrum is not favorable, or the Jordan basis ofA is ill conditioned. In the proof, some properties on the higher order derivatives of Chebyshev polynomials in an ellipse in the complex plane are derived, one of which corrects a result that has been used extensively in the literature. Supported by the China State Major Key Project for Basic Researches, the National Natural Science Foundation of China, the Doctoral Program of the Chinese National Educational Commission, the Foundation of Returned Scholars of China and Liaoning Province Natural Science Foundation.     

Solutions to Operator Equations on Hilbert C*-Modules II

In this paper, we study the solvability of the operator equations A*X + X*A = C and A*XB + B*X*A = C for general adjointable operators on Hilbert C*-modules whose ranges may not be closed. Based on these results we discuss the solution to the operator equation AXB = C, and obtain some necessary and sufficient conditions for the existence of a real positive solution, of a solution X with B*(X* + X)B ≥ 0, and of a solution X with B*XB ≥ 0. Furthermore in the special case that R(B) í [`(R(A*))]{R(B)\subseteq\overline{R(A*)}} we obtain a necessary and sufficient condition for the existence of a positive solution to the equation AXB = C. The above results generalize some recent results concerning the equations for operators with closed ranges.     

New convergence results on the global GMRES method for diagonalizable matrices

In the present paper, we give some new convergence results of the global GMRES method for multiple linear systems. In the case where the coefficient matrix A is diagonalizable, we derive new upper bounds for the Frobenius norm of the residual. We also consider the case of normal matrices and we propose new expressions for the norm of the residual.     

Homeomorphisms of universal dendrites

Homeomorphisms of some special universal dendrites are considered in the paper. In particular, dendrites are characterized for which the action onX of the group of autohomeomorphisms ofX has exactlyn≥3 orbits, and for each orbitB ofX and for each areA ⊂X the intersectionA∪B is a dense subset ofA. Some of the results generalize earlier ones from the author’s paper [2].     

A Convergence Analysis of Gmres and Fom Methods for Sylvester Equations

We discuss convergence properties of the GMRES and FOM methods for solving large Sylvester equations of the form AX–XB=C. In particular we show the importance of the separation between the fields of values of A and B on the convergence behavior of GMRES. We also discuss the stagnation phenomenon in GMRES and its consequence on FOM. We generalize the issue of breakdown in the block-Arnoldi algorithm and explain its consequence on FOM and GMRES methods. Several numerical tests illustrate the theoretical results.     

Fano 3-folds with divisible anticanonical class

We show the nonvanishing of H ⁰(X,−K _X ) for any a Fano 3-fold X for which −K _X is a multiple of another Weil divisor in Cl(X). The main case we study is Fano 3-folds with Fano index 2: that is, 3-folds X with rank Pic(X)=1, -factorial terminal singularities and −K _X  = 2A for an ample Weil divisor A. We give a first classification of all possible Hilbert series of such polarised varieties (X,A) and deduce both the nonvanishing of H ⁰(X,−K _X ) and the sharp bound (−K _X )³≥ 8/165. We find the families that can be realised in codimension up to 4.     

Numerical behaviour of the modified gram-schmidt GMRES implementation

In [6] the Generalized Minimal Residual Method (GMRES) which constructs the Arnoldi basis and then solves the transformed least squares problem was studied. It was proved that GMRES with the Householder orthogonalization-based implementation of the Arnoldi process (HHA), see [9], is backward stable. In practical computations, however, the Householder orthogonalization is too expensive, and it is usually replaced by the modified Gram-Schmidt process (MGSA). Unlike the HHA case, in the MGSA implementation the orthogonality of the Arnoldi basis vectors is not preserved near the level of machine precision. Despite this, the MGSA-GMRES performs surprisingly well, and its convergence behaviour and the ultimately attainable accuracy do not differ significantly from those of the HHA-GMRES. As it was observed, but not explained, in [6], it is thelinear independence of the Arnoldi basis, not the orthogonality near machine precision, that is important. Until the linear independence of the basis vectors is nearly lost, the norms of the residuals in the MGSA implementation of GMRES match those of the HHA implementation despite the more significant loss of orthogonality. In this paper we study the MGSA implementation. It is proved that the Arnoldi basis vectors begin to lose their linear independenceonly after the GMRES residual norm has been reduced to almost its final level of accuracy, which is proportional to κ(A)ε, where κ(A) is the condition number ofA and ε is the machine precision. Consequently, unless the system matrix is very ill-conditioned, the use of the modified Gram-Schmidt GMRES is theoretically well-justified. This work was supported by the GA AS CR under grants 230401 and A 2030706, by the NSF grant Int 921824, and by the DOE grant DEFG0288ER25053. Part of the work was performed while the third author visited Department of Mathematics and Computer Science, Emory University, Atlanta, USA.     

Group Representations with Empty Residual Spectrum

Let X be a Banach space on which a discrete group Γ acts by isometries. For certain natural choices of X, every element of the group algebra, when regarded as an operator on X, has empty residual spectrum. We show, for instance, that this occurs if X is ℓ ²(Γ) or the group von Neumann algebra VN(Γ). In our approach, we introduce the notion of a surjunctive pair, and develop some of the basic properties of this construction. The cases X = ℓ ^p (Γ) for 1 ≤ p < 2 or 2 < p < ∞ are more difficult. If Γ is amenable we can obtain partial results, using a majorization result of Herz; an example of Willis shows that some condition on Γ is necessary.     

On the existence of invariant probability measures

Let (X,A) be a measureable space andT:X →X a measurable mapping. Consider a family ℳ of probability measures onA which satisfies certain closure conditions. IfA ₀⊂A is a convergence class for ℳ such that, for everyA ∈A ₀, the sequence ((1/n) Σ _i ^=0/n−1 1 _A ∘T ⁱ) converges in distribution (with respect to some probability measurev ∈ ℳ), then there exists aT-invariant element in ℳ. In particular, for the special case of a topological spaceX and a continuous mappingT, sufficient conditions for the existence ofT-invariant Borel probability measures with additional regularity properties are obtained.     

Polaroid operators,SVEP and perturbed Browder,Weyl theorems

A Banach space operatorT ɛB(X) is polaroid,T ɛP, if the isolated points of the spectrum ofT are poles of the resolvent ofT. LetPS denote the class of operators inP which have have SVEP, the single-valued extension property. It is proved that ifT is polynomiallyPS andA ɛB(X) is an algebraic operator which commutes withT, thenf(T+A) satisfies Weyl’s theorem andf(T ^*+A ^*) satisfiesa-Weyl’s theorem for everyf which is holomorphic on a neighbourhood of σ(T+A).