BANDED TOEPLITZ PRECONDITIONERS FOR TOEPLITZ MATRICES FROM SINC METHODS

We give general expressions, analyze algebraic properties and derive eigenvalue bounds for a sequence of Toeplitz matrices associated with the sinc discretizations of various orders of differential operators. We demonstrate that these Toeplitz matrices can be satisfactorily preconditioned by certain banded Toeplitz matrices through showing that the spectra of the preconditioned matrices are uniformly bounded. In particular, we also derive eigenvalue bounds for the banded Toeplitz preconditioners. These results are elementary in constructing high-quality structured preconditioners for the systems of linear equations arising from the sinc discretizations of ordinary and partial differential equations, and are useful in analyzing algebraic properties and deriving eigenvalue bounds for the corresponding preconditioned matrices. Numerical examples are given to show effectiveness of the banded Toeplitz preconditioners.     

A unifying approach to abstract matrix algebra preconditioning

In earlier papers Tyrtyshnikov [42] and the first author [14] considered the analysis of clustering properties of the spectra of specific Toeplitz preconditioned matrices obtained by means of the best known matrix algebras. Here we generalize this technique to a generic Banach algebra of matrices by devising general preconditioners related to “convergent” approximation processes [36]. Finally, as case study, we focus our attention on the Tau preconditioning by showing how and why the best matrix algebra preconditioners for symmetric Toeplitz systems can be constructed in this class. Received April 25, 1997 / Revised version received March 13, 1998     

A Korovkin-type theory for finite Toeplitz operators via matrix algebras

Preconditioned conjugate gradients (PCG) are widely and successfully used methods for solving a Toeplitz linear system [59,9,20,5,34,62,6,10,28,45,44,46,49]. Frobenius-optimal preconditioners are chosen in some proper matrix algebras and are defined by minimizing the Frobenius distance from . The convergence features of these PCG have been naturally studied by means of the Weierstrass–Jackson Theorem [17,36,45], owing to the profound relationship between the spectral features of the matrices , generated by the Fourier coefficients of a continuous function f, and the analytical properties of the symbol f itself. In this paper, we capsize this point of view by showing that the optimal preconditioners can be used to define both new and just known linear positive operators uniformly approximating the function f. On the other hand, by modifying the Korovkin Theorem to study the Frobenius-optimal preconditioning problem, we provide a new and unifying tool for analyzing all Frobenius-optimal preconditioners in any generic matrix algebra related to trigonometric transforms. Finally, the multilevel case is sketched and discussed by showing that a Korovkin-type Theory also holds in a multivariate sense. Received October 1, 1996 / Revised version received May 7, 1998     

Regularizing properties of a class of matrices including the optimal and the superoptimal preconditioners

In this work, given a positive definite matrix A, we introduce a class of matrices related to A, which is obtained by suitably combining projections of its powers onto algebras of matrices simultaneously diagonalized by a unitary transform. After a detailed theoretical study of some spectral properties of the matrices of this class, which suggests their use as regularizing preconditioners, we prove that such matrices can be cheaply computed when the matrix A has a Toeplitz structure. We provide numerical evidence of the advantages coming from the employment of the proposed preconditioners when used in regularizing procedures.     

Inversion of Toeplitz Matrices which are not Strongly Non-singular

A type of patterned matrix called r-Toeplitz is introduced.This is more general than a block Toeplitz matrix, which resultswhen the order n of the matrix is a multiple of r.A well-knownalgorithm for inverting Toeplitz matrices is extended to dealwith r-Toeplitz matrices, involving O(rn²) operations.This enablesan algorithm to be produced for inverting Toeplitz matriceswhich are not strongly non-singular, when the standard techniquesbreak down.     

Two‐level preconditioners for serendipity finite element matrices

In this paper an approach to construct algebraic multilevel preconditioners for serendipity finite element matrices is presented. Two‐level preconditioners constructed in the paper allow to obtain multilevel preconditioners in serendipity case using multilevel preconditioners for linear finite element matrices. Copyright © 2006 John Wiley & Sons, Ltd.     

The rate of convergence of Toeplitz based PCG methods for second order nonlinear boundary value problems

Summary. In previous works [21–23] we proposed the use of [5] and band Toeplitz based preconditioners for the solution of 1D and 2D boundary value problems (BVP) by means of the preconditioned conjugate gradient (PCG) methods. As and band Toeplitz linear systems can be solved [4] by using fast sine transforms [8], these methods become especially attractive in a parallel environment of computation. In this paper we extend this technique to the nonlinear, nonsymmetric case and, in addition, we prove some clustering properties for the spectra of the preconditioned matrices showing why these methods exhibit a convergence speed which results to be more than linear. Therefore these methods work much finer than those based on separable preconditioners [18,45], on incomplete LU factorizations [36,13,27], and on circulant preconditioners [9,30,35] since the latter two techniques do not assure a linear rate of convergence. On the other hand, the proposed technique has a wider range of application since it can be naturally used for nonlinear, nonsymmetric problems and for BVP in which the coefficients of the differential operator are not strictly positive and only piecewise smooth. Finally the several numerical experiments performed here and in [22,23] confirm the effectiveness of the theoretical analysis. Received December 19, 1995 / Revised version received September 15, 1997     

Band Toeplitz preconditioners for block Toeplitz systems

We consider the solutions of block Toeplitz systems with Toeplitz blocks by the preconditioned conjugate gradient (PCG) method. Here the block Toeplitz matrices are generated by nonnegative functions f(x,y). We use band Toeplitz matrices as preconditioners. The generating functions g(x,y) of the preconditioners are trigonometric polynomials of fixed degree and are determined by minimizing (f − g)/f∞. We prove that the condition number of the preconditioned system is O(1). An a priori bound on the number of iterations for convergence is obtained.     

Circulant preconditioners for Toeplitz-block matrices

We propose two block preconditioners for Toeplitz-block matrices (i.e. each block is Toeplitz), intended to be used in conjunction with conjugate gradient methods. These preconditioners employ and extend existing circulant preconditioners for point Toeplitz matrices. The two preconditioners differ in whether the point circulant approximation is used once or twice, and also in the cost per step. We discuss efficient implementation of these two preconditioners, as well as some basic theoretical properties (such as preservation of symmetry and positive definiteness). We report results of numerical experiments, including an example from active noise control, to compare their performance.Research supported by SRI International and by the Army Research Office under contract DAAL03-91-G-0150 and by the Office of Naval Research under contract N00014-90-J-1695.     

Circulant Preconditioners for Indefinite Toeplitz Systems

In recent papers circulant preconditioners were proposed for ill-conditioned Hermitian Toeplitz matrices generated by 2-periodic continuous functions with zeros of even order. It was show that the spectra of the preconditioned matrices are uniformly bounded except for a finite number of outliers and therefore the conjugate gradient method, when applied to solving these circulant preconditioned systems, converges very quickly. In this paper, we consider indefinite Toeplitz matrices generated by 2-periodic continuous functions with zeros of odd order. In particular, we show that the singular values of the preconditioned matrices are essentially bounded. Numerical results are presented to illustrate the fast convergence of CGNE, MINRES and QMR methods.This revised version was published online in October 2005 with corrections to the Cover Date.     

Fast iterative methods for symmetric sinc-Galerkin systems

The symmetric sinc-Galerkin method developed by Lund, when appliedto the second-order self-adjoint boundary value problem, givesrise to a symmetric coefficient matrix has a special structureso that it can be advantageously used in solving the discretesystem. In this paper, we employ the preconditioned conjugategradient method with banded matrices as preconditioners. Weprove that the condition number of the preconditioned matrixis uniformly bounded by a constant independent of the size ofthe matrix. In particular, we show that the solution of an n-by-ndiscrete symmetric sinc-Galerkin system can be obtained in O(nlog n) operations. We also extend our method to the self-adjointelliptic partial differential equation. Numerical results aregiven to illustrate the effectiveness of our fast iterativesolvers.     

Preconditioners Based on Fit Techniques for the Iterative Regularization in the Image Deconvolution Problem

For large-scale image deconvolution problems, the iterative regularization methods can be favorable alternatives to the direct methods. We analyze preconditioners for regularizing gradient-type iterations applied to problems with 2D band Toeplitz coefficient matrix. For problems having separable and positive definite matrices, the fit preconditioner we have introduced in a previous paper has been shown to be effective in conjunction with CG. The cost of this preconditioner is of O(n²) operations per iteration, where n² is the pixels number of the image, whereas the cost of the circulant preconditioners commonly used for this type of problems is of O(n² log n) operations per iteration. In this paper the extension of the fit preconditioner to more general cases is proposed: namely the nonseparable positive definite case and the symmetric indefinite case. The major difficulty encountered in this extension concerns the factorization phase, where a further approximation is required. Three approximate factorizations are proposed. The preconditioners thus obtained have still a cost of O(n²) operations per iteration. A numerical experimentation shows that the fit preconditioners are competitive with the regularizing Chan preconditioner, both in the regularizing efficiency and the computational cost. AMS subject classification (2000) 65F10, 65F22.Received October 2003. Accepted December 2004. Communicated by Lars Eldén.     

A Note on Parallel Preconditioning for the All-at-Once Solution of Riesz Fractional Diffusion Equations

The $p$-step backward difference formula (BDF) for solving systems of ODEs can be formulated as all-at-once linear systems that are solved by parallel-in-time preconditioned Krylov subspace solvers (see McDonald et al. [36] and Lin and Ng [32]). However, when the BDF$p$ (2 ≤ $p$ ≤ 6) method is used to solve time-dependent PDEs, the generalization of these studies is not straightforward as $p$-step BDF is not selfstarting for $p$ ≥ 2. In this note, we focus on the 2-step BDF which is often superior to the trapezoidal rule for solving the Riesz fractional diffusion equations, and show that it results into an all-at-once discretized system that is a low-rank perturbation of a block triangular Toeplitz system. We first give an estimation of the condition number of the all-at-once systems and then, capitalizing on previous work, we propose two block circulant (BC) preconditioners. Both the invertibility of these two BC preconditioners and the eigenvalue distributions of preconditioned matrices are discussed in details. An efficient implementation of these BC preconditioners is also presented, including the fast computation of dense structured Jacobi matrices. Finally, numerical experiments involving both the one- and two-dimensional Riesz fractional diffusion equations are reported to support our theoretical findings.     

Toeplitz approximate inverse preconditioner for banded Toeplitz matrices

In this paper we introduce a new preconditioner for banded Toeplitz matrices, whose inverse is itself a Toeplitz matrix. Given a banded Hermitian positive definite Toeplitz matrixT, we construct a Toepliz matrixM such that the spectrum ofMT is clustered around one; specifically, if the bandwidth ofT is , all but eigenvalues ofMT are exactly one. Thus the preconditioned conjugate gradient method converges in +1 steps which is about half the iterations as required by other preconditioners for Toepliz systems that have been suggested in the literature. This idea has a natural extension to non-banded and non-Hermitian Toeplitz matrices, and to block Toeplitz matrices with Toeplitz blocks which arise in many two dimensional applications in signal processing. Convergence results are given for each scheme, as well as numerical experiments illustrating the good convergence properties of the new preconditioner.Partly supported by a travel fund from the Deutsche Forschungsgemeinschaft.Research supported in part by Oak Ridge Associated Universities grant no. 009707.     

Schur complement preconditioners for anisotropic problems

We present two new variants of Schur complement domain decompositionpreconditioners suitable for 2D anisotropic problems. Thesevariants are based on adaptations of the probing idea, describedby Chan et al (1992 Fifth Int. Symp. on Domain DecompositionMethods for Partial Differential Equations, Philadelphia: SIAM,pp 236-249), used in conjunction with a coarse grid approximationas introduced by Bramble et al (1986 Math. Comput. 47, 103-134).The new methods are specifically designed for situations wherethe coupling between neighbouring interfaces is stronger thanthe coupling within an interface. Taking into account this strongcoupling, one variant uses a multicolour probing technique toavoid distortion in the probe approximations that appear whenusing the method proposed by Chan et al. The second techniqueuses additional probe matrices to approximate not only the couplingwithin the interfaces but also the coupling between interfacepoints across the subdomains. This latter procedure looks somewhatlike an alternating line relaxation method for anisotropic problems,see Brandt (1977 Math. Comput.. 31, 333-390). To assess therelevance of the new preconditioners, we compare their numericalbehaviour with well known robust preconditioners such as thebalanced Neumann-Neumann method proposed by Mandel (1993 Commun.Numer. Methods Eng.. 9, 233-241).     

Structured preconditioners for nonsingular matrices of block two-by-two structures

For the large sparse block two-by-two real nonsingular matrices, we establish a general framework of practical and efficient structured preconditioners through matrix transformation and matrix approximations. For the specific versions such as modified block Jacobi-type, modified block Gauss-Seidel-type, and modified block unsymmetric (symmetric) Gauss-Seidel-type preconditioners, we precisely describe their concrete expressions and deliberately analyze eigenvalue distributions and positive definiteness of the preconditioned matrices. Also, we show that when these structured preconditioners are employed to precondition the Krylov subspace methods such as GMRES and restarted GMRES, fast and effective iteration solvers can be obtained for the large sparse systems of linear equations with block two-by-two coefficient matrices. In particular, these structured preconditioners can lead to efficient and high-quality preconditioning matrices for some typical matrices from the real-world applications.

     

Preconditioned HSS method for large multilevel block Toeplitz linear systems via the notion of matrix‐valued symbol

We perform a spectral analysis of the preconditioned Hermitian/skew‐Hermitian splitting (PHSS) method applied to multilevel block Toeplitz linear systems in which the coefficient matrix T_n(f) is associated with a Lebesgue integrable matrix‐valued function f. When the preconditioner is chosen as a Hermitian positive definite multilevel block Toeplitz matrix T_n(g), the resulting sequence of PHSS iteration matrices M_n belongs to the generalized locally Toeplitz class. In this case, we are able to compute the symbol ?(f,g) describing the asymptotic eigenvalue distribution of M_nwhen n→∞ and the matrix size diverges. By minimizing the infinity norm of the spectral radius of the symbol ?(f,g), we are also able to identify effective PHSS preconditioners T_n(g) for the matrix T_n(f). A number of numerical experiments are presented and commented, showing that the theoretical results are confirmed and that the spectral analysis leads to efficient PHSS methods. Copyright © 2015 John Wiley & Sons, Ltd.     

Effectiveness and robustness revisited for a preconditioning technique based on structured incomplete factorization

In this work, we provide new analysis for a preconditioning technique called structured incomplete factorization (SIF) for symmetric positive definite matrices. In this technique, a scaling and compression strategy is applied to construct SIF preconditioners, where off‐diagonal blocks of the original matrix are first scaled and then approximated by low‐rank forms. Some spectral behaviors after applying the preconditioner are shown. The effectiveness is confirmed with the aid of a type of two‐dimensional and three‐dimensional discretized model problems. We further show that previous studies on the robustness are too conservative. In fact, the practical multilevel version of the preconditioner has a robustness enhancement effect, and is unconditionally robust (or breakdown free) for the model problems regardless of the compression accuracy for the scaled off‐diagonal blocks. The studies give new insights into the SIF preconditioning technique and confirm that it is an effective and reliable way for designing structured preconditioners. The studies also provide useful tools for analyzing other structured preconditioners. Various spectral analysis results can be used to characterize other structured algorithms and study more general problems.