Convergence of CG and GMRES on a tridiagonal Toeplitz linear system

The Conjugate Gradient method (CG), the Minimal Residual method (MINRES), or more generally, the Generalized Minimal Residual method (GMRES) are widely used to solve a linear system Ax=b. The choice of a method depends on A’s symmetry property and/or definiteness), and MINRES is really just a special case of GMRES. This paper establishes error bounds on and sometimes exact expressions for residuals of CG, MINRES, and GMRES on solving a tridiagonal Toeplitz linear system, where A is Hermitian or just normal. These expressions and bounds are in terms of the three parameters that define A and Chebyshev polynomials of the first or second kind. AMS subject classification (2000)  65F10, 65N22     

A Class of Incomplete Orthogonal Factorization Methods. II: Implementation and Results

We present, implement and test several incomplete QR factorization methods based on Givens rotations for sparse square and rectangular matrices. For square systems, the approximate QR factors are used as right-preconditioners for GMRES, and their performance is compared to standard ILU techniques. For rectangular matrices corresponding to linear least-squares problems, the approximate R factor is used as a right-preconditioner for CGLS. A comprehensive discussion is given about the uses, advantages and shortcomings of the preconditioners. AMS subject classification (2000) 65F10, 65F25, 65F50.Received May 2002. Revised October 2004. Communicated by Åke Björck.     

CGS/GMRES(k): AN ADAPTIVE PRECONDITIONED CGS ALGORITHM FOR NONSYMMETRIC LINEAR SYSTEMS

Recently Y. Saad proposed a flexible inner-outer preconditioned GMRES algorithm for nonsymmetric linear systems [4]. Following their ideas, we suggest an adaptive preconditioned CGS method, called CGS/GMRES (k), in which the preconditioner is constructed in the iteration step of CGS, by several steps of GMRES(k). Numerical experiments show that the residual of the outer iteration decreases rapidly. We also found the interesting residual behaviour of GMRES for the skewsymmetric linear system Ax = b, which gives a convergence result for restarted GMRES (k). For convenience, we discuss real systems.     

Iterative regularization with minimum-residual methods

We study the regularization properties of iterative minimum-residual methods applied to discrete ill-posed problems. In these methods, the projection onto the underlying Krylov subspace acts as a regularizer, and the emphasis of this work is on the role played by the basis vectors of these Krylov subspaces. We provide a combination of theory and numerical examples, and our analysis confirms the experience that MINRES and MR-II can work as general regularization methods. We also demonstrate theoretically and experimentally that the same is not true, in general, for GMRES and RRGMRES – their success as regularization methods is highly problem dependent. AMS subject classification (2000)  65F22, 65F10     

Hopf-Takens-Bogdanov interaction for a fluid-conveying tube

We investigate the dynamics after loss of stability of the downhanging configuration of a fluid conveying tube with a small end mass and an elastic support. By varying the fluid flow rate and the stiffness and location of the elastic support, different degenerate bifurcation scenarios can be observed. In this article we investigate the bifurcating solution branches of the codimension 3 interaction between a Hopf bifurcation and a Bogdanov-Takens bifurcation. A complete discussion of the primary and secondary solution branches was already given by W. F. Langford and K. Zhan. After reducing the system to the three-dimensional Normal Form equations we apply a numerical continuation procedure to locate the expected higher order bifurcation branches and detect more complicated dynamics, like Shilnikov orbits. (© 2016 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

A PRODUCT HYBRID GMRES ALGORITHM FOR NONSYMMETRIC LINEAR SYSTEMS

It has been observed that the residual polynomials resulted from successive restarting cycles of GMRES(m) may differ from one another meaningfully. In this paper, it is further shown that the polynomials can complement one another harmoniously in reducing the iterative residual. This characterization of GMRES(m) is exploited to formulate an efficient hybrid iterative scheme, which can be widely applied to existing hybrid algorithms for solving large nonsymmetric systems of linear equations. In particular, a variant of the hybrid GMRES algorithm of Nachtigal, Reichel and Trefethen (1992) is presented. It is described how the new algorithm may offer significant performance improvements over the original one.     

Numerical experiments on a flexible variant of GMRES-DR

The Flexible GMRES (FGMRES [1]) and the GMRES with deflated restarting (GMRES-DR [2]) methods are two algorithms derived from GMRES [3], that are considered as powerful when solving large non hermitian systems of linear equations. GMRES-DR is a variant of GMRES with an improved restarting technique that maintains in the Krylov subspace harmonic Ritz vector from the previous restart. In situations where the convergence of restarted GMRES is slow and where the matrix has few eigenvalues close to the origin, this technique has proved very efficient. The new method that we propose is the Flexible GMRES with deflated restarting (FGMRES-DR [6]), which combines the two above mentioned algorithms in order to yield better performance. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Preconditioners for nonsymmetric block toeplitz-like-plus-diagonal linear systems

Summary. We consider the system of linear equations Lu=f, where L is a nonsymmetric block Toeplitz-like-plus-diagonal matrix, which arises from the Sinc-Galerkin discretization of differential equations. Our main contribution is to construct effective preconditioners for this structured coefficient matrix and to derive tight bounds for eigenvalues of the preconditioned matrices. Moreover, we use numerical examples to show that the new preconditioners, when applied to the preconditioned GMRES method, are efficient for solving the system of linear equations. Mathematics Subject Classification (2000):65F10, 65F15, 65T10Research subsidized by The Special Funds for Major State Basic Research Projects G1999032803Research supported in part by RGC Grant Nos. 7132/00P and 7130/02P, and HKU CRCG Grant Nos 10203501, 10203907 and 10203408     

Partitioned-GMRES in Domain Decomposition with Approximate Subdomain Solution

Solution of large linear systems encountered in computational fluid dynamics often leads to some form of domain decomposition, especially when it is desired to use parallel machines. In this paper P-GMRES, a partitioned modification of GMRES, is applied to such problems. It is shown that P-GMRES converges faster than GMRES if the subdomains are solved exactly, and that P-GMRES requires less communication in the computation of the inner products. Also, approximate solutions for the subdomains by an inner preconditioned GMRES iteration are considered, in combination with a restarted version of P-GMRES. It turns out that rather crude tolerances are allowed, and that a good strategy is to vary the tolerance for the subdomains in the course of the outer iteration.This revised version was published online in October 2005 with corrections to the Cover Date.     

A simpler GMRES and its adaptive variant for shifted linear systems

A variant of the simpler GMRES method is developed for solving shifted linear systems (SGMRES‐Sh), exhibiting almost the same advantage of the simpler GMRES method over the regular GMRES method. Because the remedy adapted by GMRES‐Sh is no longer feasible for SGMRES‐Sh due to the differences between simpler GMRES and GMRES for constructing the residual vectors of linear systems, we take an alternative strategy to force the residual vectors of the add system also be orthogonal to the subspaces, to which the residual vectors of the seed system are orthogonal when the seed system is solved with the simpler GMRES method. In addition, a seed selection strategy is also employed for solving the rest non‐converged linear systems. Furthermore, an adaptive version of SGMRES‐Sh is presented for the purpose of improving the stability of SGMRES‐Sh based on the technique of the adaptive choice of the Krylov subspace basis developed for the adaptive simpler GMRES. Numerical experiments demonstrate the benefits of the presented methods.     

Approximate Inverse Preconditioners for Some Large Dense Random Electrostatic Interaction Matrices

A sparse mesh-neighbour based approximate inverse preconditioner is proposed for a type of dense matrices whose entries come from the evaluation of a slowly decaying free space Green’s function at randomly placed points in a unit cell. By approximating distant potential fields originating at closely spaced sources in a certain way, the preconditioner is given properties similar to, or better than, those of a standard least squares approximate inverse preconditioner while its setup cost is only that of a diagonal block approximate inverse preconditioner. Numerical experiments on iterative solutions of linear systems with up to four million unknowns illustrate how the new preconditioner drastically outperforms standard approximate inverse preconditioners of otherwise similar construction, and especially so when the preconditioners are very sparse. AMS subject classification (2000) 65F10, 65R20, 65F35, 78A30     

基于GMRES的多项式预处理广义极小残差法

求解大型稀疏线性方程组一般采用迭代法,其中GMRES(m)算法是一种非常有效的算法,然而该算法在解方程组时,可能发生停滞．为了克服算法GMRES(m)解线性系统Ax=b过程中可能出现的收敛缓慢或不收敛,本文利用GMRES本身构造出一种有效的多项式预处理因子pk(z),该多项式预处理因子非常简单且易于实现．数值试验表明,新算法POLYGMRES(m)较好地克服了GMRES(m)的缺陷．     

Linear least squares problems with data over incomplete grids

The paper addresses bivariate surface fitting problems, where data points lie on the vertices of a rectangular grid. Efficient and stable algorithms can be found in the literature to solve such problems. If data values are missing at some grid points, there exists a computational method for finding a least squares spline by fixing appropriate values for the missing data. We extended this technique to arbitrary least squares problems as well as to linear least squares problems with linear equality constraints. Numerical examples are given to show the effectiveness of the technique presented. AMS subject classification (2000)  65D05, 65D07, 65D10, 65F05, 65F20     

CMRH: A new method for solving nonsymmetric linear systems based on the Hessenberg reduction algorithm

The Generalized Minimal Residual (GMRES) method and the Quasi-Minimal Residual (QMR) method are two Krylov methods for solving linear systems. The main difference between these methods is the generation of the basis vectors for the Krylov subspace. The GMRES method uses the Arnoldi process while QMR uses the Lanczos algorithm for constructing a basis of the Krylov subspace. In this paper we give a new method similar to QMR but based on the Hessenberg process instead of the Lanczos process. We call the new method the CMRH method. The CMRH method is less expensive and requires slightly less storage than GMRES. Numerical experiments suggest that it has behaviour similar to GMRES. This revised version was published online in June 2006 with corrections to the Cover Date.     

Multiresolution weighted norm equivalences and applications

Summary. We establish multiresolution norm equivalences in weighted spaces L ² _w ((0,1)) with possibly singular weight functions w(x)≥0 in (0,1). Our analysis exploits the locality of the biorthogonal wavelet basis and its dual basis functions. The discrete norms are sums of wavelet coefficients which are weighted with respect to the collocated weight function w(x) within each scale. Since norm equivalences for Sobolev norms are by now well-known, our result can also be applied to weighted Sobolev norms. We apply our theory to the problem of preconditioning p-Version FEM and wavelet discretizations of degenerate elliptic and parabolic problems from finance. Revised version received March 19, 2003 Mathematics Subject Classification (2000): 65F35, 65F50, 65N22, 65N35, 65N30, 65T60, 60H10, 60H35 An erratum to this article is available at .     

Krylov子空间投影法及其在油藏数值模拟中的应用

Krylov子空间投影法是一类非常有效的大型线性代数方程组解法,随着左右空间L_m、K_m的不同选取可以得到许多人们熟知的方法.按矩阵H_m的不同类型,将Krylov子空间方法分成两大类,简要分析了这两类方法的优缺点及其最新进展.将目前最为可靠实用的广义最小余量法(GMRES)应用于油藏数值模拟计算问题,利用矩阵分块技术,采用块拟消去法(PE)对系数阵进行预处理.计算结果表明本文的预处理GMRES方法优于目前使用较多的预处理正交极小化ORTHMIN方法,最后还讨论了投影类方法的局限和今后的可能发展方向.     

BISTAB: A portable bifurcation and stability analysis package

A portable bifurcation and stability analysis package, called BISTAB, is described. The package is written in FORTRAN V and can follow the connected set of equilibrium curves for a system of nonlinear ordinary differential equations in the state × parameter space by varying a bifurcation parameter. The curves are traced from an initial point with the continuation method of Kubicek, and the tangent method of Keller is used to find initial points on bifurcating curves near simple bifurcation points. Linearized stability analysis, location of Hopf bifurcation points, and sorting of points for plotting are also supported. While the package contains no new numerical methods, the lack of a requirement for any derivative information higher than the Jacobian makes BISTAB computationally efficient and useful for applied problems where nonnumerical bifurcation analysis may be difficult.     

Krylov subspace methods with deflation and balancing preconditioners for least squares problems

For solving least squares problems, the CGLS method is a typical method in the point of view of iterative methods. When the least squares problems are ill-conditioned, the convergence behavior of the CGLS method will present a deteriorated result. We expect to select other iterative Krylov subspace methods to overcome the disadvantage of CGLS. Here the GMRES method is a suitable algorithm for the reason that it is derived from the minimal residual norm approach, which coincides with least squares problems. Ken Hayami proposed BAGMRES for solving least squares problems in [\emph{GMRES Methods for Least Squares Problems, SIAM J. Matrix Anal. Appl., 31(2010)}, pp.2400-2430]. The deflation and balancing preconditioners can optimize the convergence rate through modulating spectral distribution. Hence, in this paper we utilize preconditioned iterative Krylov subspace methods with deflation and balancing preconditioners in order to solve ill-conditioned least squares problems. Numerical experiments show that the methods proposed in this paper are better than the CGLS method.     

Numerical stability analysis and computation of Hopf bifurcation points for delay differential equations

We present a numerical technique for the stability analysis and the computation of branches of Hopf bifurcation points in nonlinear systems of delay differential equations with several constant delays. The stability analysis of a steady-state solution is done by a numerical implementation of the argument principle, which allows to compute the number of eigenvalues with positive real part of the characteristic matrix. The technique is also used to detect bifurcations of higher singularity (Hopf and fold bifurcations) during the continuation of a branch of Hopf points. This allows to trace new branches of Hopf points and fold points.     

Application of a Krylov subspace iterative method in a multi-level adaptive technique to solve the mild-slope equation in nearshore regions

A multi-level adaptive numerical technique is applied to a nonlinear formulation of the mild-slope equation, to obtain the nearshore wave field, where the dominant processes of wave transformation are shoaling, refraction and diffraction. The advantage of this formulation over the traditional elliptic, parabolic and hyperbolic formulations is to require a lower minimum number of grid nodes per wavelength, thus, its capacity to predict the wave field for larger coastal areas. The efficiency of the interactions between the grid mesh levels, where two robust Krylov subspace iterative methods, the Bi-CGSTAB and the GMRES, are applied to solve the governing equation, is tested, for several hierarchies of grid mesh levels. The results show that the multi-level adaptive technique is efficient only if the GMRES iterative method is applied, and that for six grid mesh levels good results can be achieved for a residual as low as 10⁻³ for the finest grid.