Transpose-free formulations of Lanczos-type methods for nonsymmetric linear systems

We present a transpose-free version of the nonsymmetric scaled Lanczos procedure. It generates the same tridiagonal matrix as the classical algorithm, using two matrix–vector products per iteration without accessing A^T. We apply this algorithm to obtain a transpose-free version of the Quasi-minimal residual method of Freund and Nachtigal [15] (without look-ahead), which requires three matrix–vector products per iteration. We also present a related transpose-free version of the bi-conjugate gradients algorithm. This revised version was published online in June 2006 with corrections to the Cover Date.     

Nested Lanczos: implicitly restarting an unsymmetric Lanczos algorithm

In this text, we present a generalization of the idea of the Implicitly Restarted Arnoldi method to the unsymmetric Lanczos algorithm, using the two-sided Gram-Schmidt process or using a full Lanczos tridiagonalization. The resulting implicitly restarted Lanczos method is called Nested Lanczos. Nested Lanczos can be combined with an implicit filter. It can also be used in case of breakdown and offers an alternative for look-ahead. This revised version was published online in August 2006 with corrections to the Cover Date.     

A look-ahead algorithm for the solution of general Hankel systems

Summary The solution of systems of linear equations with Hankel coefficient matrices can be computed with onlyO(n ²) arithmetic operations, as compared toO(n ³) operations for the general cases. However, the classical Hankel solvers require the nonsingularity of all leading principal submatrices of the Hankel matrix. The known extensions of these algorithms to general Hankel systems can handle only exactly singular submatrices, but not ill-conditioned ones, and hence they are numerically unstable. In this paper, a stable procedure for solving general nonsingular Hankel systems is presented, using a look-ahead technique to skip over singular or ill-conditioned submatrices. The proposed approach is based on a look-ahead variant of the nonsymmetric Lanczos process that was recently developed by Freund, Gutknecht, and Nachtigal. We first derive a somewhat more general formulation of this look-ahead Lanczos algorithm in terms of formally orthogonal polynomials, which then yields the look-ahead Hankel solver as a special case. We prove some general properties of the resulting look-ahead algorithm for formally orthogonal polynomials. These results are then utilized in the implementation of the Hankel solver. We report some numerical experiments for Hankel systems with ill-conditioned submatrices.The research of the first author was supported by DARPA via Cooperative Agreement NCC 2-387 between NASA and the Universities Space Research Association (USRA).The research of the second author was supported in part by NSF grant DRC-8412314 and Cooperative Agreement NCC 2-387 between NASA and the Universities Space Research Association (USRA).     

Chebyshev rational interpolation

The Lanczos method and its variants can be used to solve efficiently the rational interpolation problem. In this paper we present a suitable fast modification of a general look-ahead version of the Lanczos process in order to deal with polynomials expressed in the Chebyshev orthogonal basis. The proposed approach is particularly suited for approximating analytic functions by means of rational interpolation at certain nodes located on the boundary of an elliptical region of the complex plane. In fact, in this case it overcomes some of the numerical difficulties which limited the applicability of the look-ahead Lanczos process for determining the coefficients both of the numerators and of the denominators with respect to the standard power basis. This revised version was published online in August 2006 with corrections to the Cover Date.     

A Lanczos-type method for multiple starting vectors

Given a square matrix and single right and left starting vectors, the classical nonsymmetric Lanczos process generates two sequences of biorthogonal basis vectors for the right and left Krylov subspaces induced by the given matrix and vectors. In this paper, we propose a Lanczos-type algorithm that extends the classical Lanczos process for single starting vectors to multiple starting vectors. Given a square matrix and two blocks of right and left starting vectors, the algorithm generates two sequences of biorthogonal basis vectors for the right and left block Krylov subspaces induced by the given data. The algorithm can handle the most general case of right and left starting blocks of arbitrary sizes, while all previously proposed extensions of the Lanczos process are restricted to right and left starting blocks of identical sizes. Other features of our algorithm include a built-in deflation procedure to detect and delete linearly dependent vectors in the block Krylov sequences, and the option to employ look-ahead to remedy the potential breakdowns that may occur in nonsymmetric Lanczos-type methods.

     

A linear system solver based on a modified Krylov subspace method for breakdown recovery

Despite its usefulness in solving eigenvalue problems and linear systems of equations, the nonsymmetric Lanczos method is known to suffer from a potential breakdown problem. Previous and recent approaches for handling the Lanczos exact and near-breakdowns include, for example, the look-ahead schemes by Parlett-Taylor-Liu [23], Freund-Gutknecht-Nachtigal [9], and Brezinski-Redivo Zaglia-Sadok [4]; the combined look-ahead and restart scheme by Joubert [18]; and the low-rank modified Lanczos scheme by Huckle [17]. In this paper, we present yet another scheme based on a modified Krylov subspace approach for the solution of nonsymmetric linear systems. When a breakdown occurs, our approach seeks a modified dual Krylov subspace, which is the sum of the original subspace and a new Krylov subspaceK _m (w _j ,A ^T ) wherew _j is a newstart vector (this approach has been studied by Ye [26] for eigenvalue computations). Based on this strategy, we have developed a practical algorithm for linear systems called the MLAN/QM algorithm, which also incorporates the residual quasi-minimization as proposed in [12]. We present a few convergence bounds for the method as well as numerical results to show its effectiveness.Research supported by Natural Sciences and Engineering Research Council of Canada.     

On some structured inverse eigenvalue problems

This work deals with various finite algorithms that solve two special Structured Inverse Eigenvalue Problems (SIEP). The first problem we consider is the Jacobi Inverse Eigenvalue Problem (JIEP): given some constraints on two sets of reals, find a Jacobi matrix J (real, symmetric, tridiagonal, with positive off-diagonal entries) that admits as spectrum and principal subspectrum the two given sets. Two classes of finite algorithms are considered. The polynomial algorithm which is based on a special Euclid–Sturm algorithm (Householder's terminology) and has been rediscovered several times. The matrix algorithm which is a symmetric Lanczos algorithm with a special initial vector. Some characterization of the matrix ensures the equivalence of the two algorithms in exact arithmetic. The results of the symmetric situation are extended to the nonsymmetric case. This is the second SIEP to be considered: the Tridiagonal Inverse Eigenvalue Problem (TIEP). Possible breakdowns may occur in the polynomial algorithm as it may happen with the nonsymmetric Lanczos algorithm. The connection between the two algorithms exhibits a similarity transformation from the classical Frobenius companion matrix to the tridiagonal matrix. This result is used to illustrate the fact that, when computing the eigenvalues of a matrix, the nonsymmetric Lanczos algorithm may lead to a slow convergence, even for a symmetric matrix, since an outer eigenvalue of the tridiagonal matrix of order n − 1 can be arbitrarily far from the spectrum of the original matrix. This revised version was published online in August 2006 with corrections to the Cover Date.     

A generalization of biorthogonal systems based on Drazin inverse

In this paper, the transformations of generating systems of Euclidean space are examined in connection with the Drazin inverse of their Gram matrices and the notion of biorthogonal systems is generalized. The results obtained in this paper can be considered as the extension of some recent results concerning Moore-Penrose biorthogonal systems and reflexiveg-inverse systems in Euclidean spaces.     

Recurrence relation for biorthogonal polynomials

We use a geometric approach to obtain a recurrence relation for two families of biorthogonal polynomials associated to a nonsingular, strongly regular matrix M. We propose a “look-ahead procedure” for computing the biorthogonal polynomials when M has singular or ill-conditioned leading principal submatrices. These polynomials lead to two recursive triangular factorizations for the inverse of a nonsingular matrix M which is not necessarily strongly regular.     

A look-ahead strategy for the implementation of some old and new extrapolation methods

Sequence transformations, used for accelerating the convergence, are related to biorthogonal polynomials. In the particular cases of theG-transformation and the Shanks transformation (that is the -algorithm of Wynn), there is a connection with formal orthogonal polynomials. In this paper, this connection is exploited in order to propose a look-ahead strategy for the implementation of these two transformations. This strategy, which is quite similar to the strategy used for treating the same type of problems in Lanczos-based methods for solving systems of linear equations, consists in jumping over the polynomials which do not exist, thus avoiding a division by zero (breakdown) in the algorithms, and over those which could be badly computed (near-breakdown) thus leading to a better numerical stability. Numerical examples illustrate the procedure.     

The stable computation of formal orthogonal polynomials

For many applications — such as the look-ahead variants of the Lanczos algorithm — a sequence of formal (block-)orthogonal polynomials is required. Usually, one generates such a sequence by taking suitable polynomial combinations of a pair of basis polynomials. These basis polynomials are determined by a look-ahead generalization of the classical three term recurrence, where the polynomial coefficients are obtained by solving a small system of linear equations. In finite precision arithmetic, the numerical orthogonality of the polynomials depends on a good choice of the size of the small systems; this size is usually controlled by a heuristic argument such as the condition number of the small matrix of coefficients. However, quite often it happens that orthogonality gets lost.We present a new variant of the Cabay-Meleshko algorithm for numerically computing pairs of basis polynomials, where the numerical orthogonality is explicitly monitored with the help of stability parameters. A corresponding error analysis is given. Our stability parameter is shown to reflect the condition number of the underlying Hankel matrix of moments. This enables us to prove the weak and strong stability of our method, provided that the corresponding Hankel matrix is well-conditioned.This work was partially supported by the HCM project ROLLS, under contract CHRX-CT93-0416.     

N–Term Approximation in Anisotropic Function Spaces

In L₂(0, 1)²) infinitely many different biorthogonal wavelet bases may be introduced by taking tensor products of one–dimensional biorthogonal wavelet bases on the interval (0, 1). Most well–known are the standard tensor product bases and the hyperbolic bases. In [23, 24] further biorthogonal wavelet bases are introduced, which provide wavelet characterizations for functions in anisotropic Besov spaces. Here we address the following question: Which of those biorthogonal tensor product wavelet bases is the most appropriate one for approximating nonlinearly functions from anisotropic Besov spaces? It turns out, that the hyperbolic bases lead to nonlinear algorithms which converge as fast as the corresponding schemes with respect to specific anisotropy adapted bases.     

Computing the generalized singular values/vectors of large sparse or structured matrix pairs

Summary. We present a numerical algorithm for computing a few extreme generalized singular values and corresponding vectors of a sparse or structured matrix pair . The algorithm is based on the CS decomposition and the Lanczos bidiagonalization process. At each iteration step of the Lanczos process, the solution to a linear least squares problem with as the coefficient matrix is approximately computed, and this consists the only interface of the algorithm with the matrix pair . Numerical results are also given to demonstrate the feasibility and efficiency of the algorithm. Received April 1, 1994 / Revised version received December 15, 1994     

Quasi-interpolatory refinable functions and construction of biorthogonal wavelet systems

We present a new family of compactly supported and symmetric biorthogonal wavelet systems. Each refinement mask in this family has tension parameter ω. When ω = 0, it becomes the minimal length biorthogonal Coifman wavelet system (Wei et al., IEEE Trans Image Proc 7:1000–1013, 1998). Choosing ω away from zero, we can get better smoothness of the refinable functions at the expense of slightly larger support. Though the construction of the new biorthogonal wavelet systems, in fact, starts from a new class of quasi-interpolatory subdivision schemes, we find that the refinement masks accidently coincide with the ones by Cohen et al. (Comm Pure Appl Math 45:485–560, 1992, §6.C) (or Daubechies 1992, §8.3.5), which are designed for the purpose of generating biorthogonal wavelets close to orthonormal cases. However, the corresponding mathematical analysis is yet to be provided. In this study, we highlight the connection between the quasi-interpolatory subdivision schemes and the masks by Cohen, Daubechies and Feauveau, and then we study the fundamental properties of the new biorthogonal wavelet systems such as regularity, stability, linear independence and accuracy.     

Error analysis of symplectic Lanczos method for Hamiltonian eigenvalue problem

A rounding error analysis for the symplectic Lanczos method is given to solve the large-scale sparse Hamiltonian eigenvalue problem. If no breakdown occurs in the method, then it can be shown that the Hamiltonian structure preserving requirement does not destroy the essential feature of the nonsymmetric Lanczos algorithm. The relationship between the loss of J-orthogonality among the symplectic Lanczos vectors and the convergence of the Ritz values in the symmetric Lanczos algorithm is discussed. It is demonstrated that under certain assumptions the computed J-orthogonal Lanczos vectors lose the J-orthogonality when some Ritz values begin to converge. Our analysis closely follows the recent works of Bai and Fabbender. Selected from Journal of Mathematical Research and Exposition, 2004, 24(1): 91–106     

Strategies for spectrum slicing based on restarted Lanczos methods

In the context of symmetric-definite generalized eigenvalue problems, it is often required to compute all eigenvalues contained in a prescribed interval. For large-scale problems, the method of choice is the so-called spectrum slicing technique: a shift-and-invert Lanczos method combined with a dynamic shift selection that sweeps the interval in a smart way. This kind of strategies were proposed initially in the context of unrestarted Lanczos methods, back in the 1990’s. We propose variations that try to incorporate recent developments in the field of Krylov methods, including thick restarting in the Lanczos solver and a rational Krylov update when moving from one shift to the next. We discuss a parallel implementation in the SLEPc library and provide performance results.     

Optimized look-ahead recurrences for adjacent rows in the Padé table

A new look-ahead algorithm for recursively computing Padé approximants is introduced. It generates a subsequence of the Padé approximants on two adjacent rows (defined by fixed numerator degree) of the Padé table. Its two basic versions reduce to the classical Levinson and Schur algorithms if no look-ahead is required. The new algorithm can be viewed as a combination of the look-ahead sawtooth and the look-ahead Levinson and Schur algorithms that we proposed before, but now the look-ahead step size is minimal (as in the sawtooth version) and the computational costs are as low as in the least expensive competing algorithms (including our look-ahead Levinson and Schur algorithms). The underlying recurrences link well-conditioned basic pairs,i.e., pairs of sufficiently different neighboring Padé forms.The algorithm can be used to solve Toeplitz systems of equationsTx = b. In this application it comes in several versions: anO(N ²) Levinson-type form, anO(N ²) Schur-type form, and a superfastO(N log² N) Schur-type version. As an option of the first two versions, the corresponding block LDU decompositions ofT ^–1 orT, respectively, can be found.     

Discrete wavelets and the Vilenkin-Chrestenson transform

In the spaces of complex periodic sequences, we use the Vilenkin-Chrestenson transforms to construct new orthogonal wavelet bases defined by finite collections of parameters. Earlier similar bases were defined for the Cantor and Vilenkin groups by means of generalized Walsh functions. It is noted that similar constructions can be realized for biorthogonal wavelets as well as for the space ? ²(?₊).     

Fast generalized cross validation using Krylov subspace methods

The task of fitting smoothing spline surfaces to meteorological data such as temperature or rainfall observations is computationally intensive. The generalized cross validation (GCV) smoothing algorithm, if implemented using direct matrix techniques, is O(n ³) computationally, and memory requirements are O(n ²). Thus, for data sets larger than a few hundred observations, the algorithm is prohibitively slow. The core of the algorithm consists of solving series of shifted linear systems, and iterative techniques have been used to lower the computational complexity and facilitate implementation on a variety of supercomputer architectures. For large data sets though, the execution time is still quite high. In this paper we describe a Lanczos based approach that avoids explicitly solving the linear systems and dramatically reduces the amount of time required to fit surfaces to sets of data.      

基于Lanczos方法的结构动力学灵敏度分析

利用数学理论得到结构动力学重特征值的灵敏度表达式,从而解决了奇异性问题.然后,为降低计算工作量,基于Lanczos方法得到降阶的结构系统,从而得到降阶的灵敏度分析近似解.用一个算例证明的方法正确性.