Numerical methods for finding multiple eigenvalues of matrices depending on parameters

Summary. Let be a square matrix dependent on parameters and , of which we choose as the eigenvalue parameter. Many computational problems are equivalent to finding a point such that has a multiple eigenvalue at . An incomplete decomposition of a matrix dependent on several parameters is proposed. Based on the developed theory two new algorithms are presented for computing multiple eigenvalues of with geometric multiplicity . A third algorithm is designed for the computation of multiple eigenvalues with geometric multiplicity but which also appears to have local quadratic convergence to semi-simple eigenvalues. Convergence analyses of these methods are given. Several numerical examples are presented which illustrate the behaviour and applications of our methods. Received December 19, 1994 / Revised version received January 18, 1996     

Numerical enclosure for multiple eigenvalues of an Hermitian matrix whose graph is a tree

In this paper we consider a numerical enclosure method for multiple eigenvalues of an Hermitian matrix whose graph is a tree. If an Hermitian matrix A whose graph is a tree has multiple eigenvalues, it has the property that matrices which are associated with some branches in the undirected graph of A have the same eigenvalues. By using this property and interlacing inequalities for Hermitian matrices, we show an enclosure method for multiple eigenvalues of an Hermitian matrix whose graph is a tree. Since we do not generally know whether a given matrix has exactly a multiple eigenvalue from approximate computations, we use the property of interlacing inequalities to enclose some eigenvalues including multiplicities.In this process, we only use the enclosure of simple eigenvalues to enclose a multiple eigenvalue by using a computer and interval arithmetic.     

Perturbation bounds for means of eigenvalues and invariant subspaces

When a matrix is close to a matrix with a multiple eigenvalue, the arithmetic mean of a group of eigenvalues is a good approximation to this multiple eigenvalue. A theorem of Gershgorin type for means of eigenvalues is proved and applied as a perturbation theorem for a degenerate matrix.For a multiple eigenvalue we derive bounds for computed bases of subspaces of eigenvectors and principal vectors, relating them to the spaces spanned by the last singular vectors of corresponding powers of the matrix. These bounds assure that, provided the dimensionalities are chosen appropriately, the angles of rotation of the subspaces are of the same order of magnitude as the perturbation of the matrix.A numerical example is given.     

The integral graphs with index 3 and exactly two main eigenvalues

A graph is called integral if the spectrum of its adjacency matrix has only integral eigenvalues. An eigenvalue of a graph is called main eigenvalue if it has an eigenvector such that the sum of whose entries is not equal to zero. In this paper, we show that there are exactly 25 connected integral graphs with exactly two main eigenvalues and index 3.     

A block Newton method for nonlinear eigenvalue problems

We consider matrix eigenvalue problems that are nonlinear in the eigenvalue parameter. One of the most fundamental differences from the linear case is that distinct eigenvalues may have linearly dependent eigenvectors or even share the same eigenvector. This has been a severe hindrance in the development of general numerical schemes for computing several eigenvalues of a nonlinear eigenvalue problem, either simultaneously or subsequently. The purpose of this work is to show that the concept of invariant pairs offers a way of representing eigenvalues and eigenvectors that is insensitive to this phenomenon. To demonstrate the use of this concept in the development of numerical methods, we have developed a novel block Newton method for computing such invariant pairs. Algorithmic aspects of this method are considered and a few academic examples demonstrate its viability.     

The inverse eigenvalue problem for symmetric quasi anti-bidiagonal matrices

In this paper we construct the symmetric quasi anti-bidiagonal matrix that its eigenvalues are given, and show that the problem is also equivalent to the inverse eigenvalue problem for a certain symmetric tridiagonal matrix which has the same eigenvalues. Not only elements of the tridiagonal matrix come from quasi anti-bidiagonal matrix, but also the places of elements exchange based on some conditions.     

A curvilinear optimization method based upon iterative estimation of the eigensystem of the Hessian matrix

An algorithm was recently presented that minimizes a nonlinear function in several variables using a Newton-type curvilinear search path. In order to determine this curvilinear search path the eigenvalue problem of the Hessian matrix of the objective function has to be solved at each iteration of the algorithm. In this paper an iterative procedure requiring gradient information only is developed for the approximation of the eigensystem of the Hessian matrix. It is shown that for a quadratic function the approximated eigenvalues and eigenvectors tend rapidly to the actual eigenvalues and eigenvectors of its Hessian matrix. The numerical tests indicate that the resulting algorithm is very fast and stable. Moreover, the fact that some approximations to the eigenvectors of the Hessian matrix are available is used to get past saddle points and accelerate the rate of convergence on flat functions.     

The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices

We consider the eigenvalues and eigenvectors of finite, low rank perturbations of random matrices. Specifically, we prove almost sure convergence of the extreme eigenvalues and appropriate projections of the corresponding eigenvectors of the perturbed matrix for additive and multiplicative perturbation models.The limiting non-random value is shown to depend explicitly on the limiting eigenvalue distribution of the unperturbed random matrix and the assumed perturbation model via integral transforms that correspond to very well-known objects in free probability theory that linearize non-commutative free additive and multiplicative convolution. Furthermore, we uncover a phase transition phenomenon whereby the large matrix limit of the extreme eigenvalues of the perturbed matrix differs from that of the original matrix if and only if the eigenvalues of the perturbing matrix are above a certain critical threshold. Square root decay of the eigenvalue density at the edge is sufficient to ensure that this threshold is finite. This critical threshold is intimately related to the same aforementioned integral transforms and our proof techniques bring this connection and the origin of the phase transition into focus. Consequently, our results extend the class of ‘spiked’ random matrix models about which such predictions (called the BBP phase transition) can be made well beyond the Wigner, Wishart and Jacobi random ensembles found in the literature. We examine the impact of this eigenvalue phase transition on the associated eigenvectors and observe an analogous phase transition in the eigenvectors. Various extensions of our results to the problem of non-extreme eigenvalues are discussed.     

Krylov eigenvalue strategy using the FEAST algorithm with inexact system solves

The FEAST eigenvalue algorithm is a subspace iteration algorithm that uses contour integration to obtain the eigenvectors of a matrix for the eigenvalues that are located in any user‐defined region in the complex plane. By computing small numbers of eigenvalues in specific regions of the complex plane, FEAST is able to naturally parallelize the solution of eigenvalue problems by solving for multiple eigenpairs simultaneously. The traditional FEAST algorithm is implemented by directly solving collections of shifted linear systems of equations; in this paper, we describe a variation of the FEAST algorithm that uses iterative Krylov subspace algorithms for solving the shifted linear systems inexactly. We show that this iterative FEAST algorithm (which we call IFEAST) is mathematically equivalent to a block Krylov subspace method for solving eigenvalue problems. By using Krylov subspaces indirectly through solving shifted linear systems, rather than directly using them in projecting the eigenvalue problem, it becomes possible to use IFEAST to solve eigenvalue problems using very large dimension Krylov subspaces without ever having to store a basis for those subspaces. IFEAST thus combines the flexibility and power of Krylov methods, requiring only matrix–vector multiplication for solving eigenvalue problems, with the natural parallelism of the traditional FEAST algorithm. We discuss the relationship between IFEAST and more traditional Krylov methods and provide numerical examples illustrating its behavior.     

Verified numerical computations for multiple and nearly multiple eigenvalues of elliptic operators

In this paper, we propose a numerical method to verify bounds for multiple eigenvalues for elliptic eigenvalue problems. We calculate error bounds for approximations of multiple eigenvalues and base functions of the corresponding invariant subspaces. For matrix eigenvalue problems, Rump (Linear Algebra Appl. 324 (2001) 209) recently proposed a validated numerical method to compute multiple eigenvalues. In this paper, we extend his formulation to elliptic eigenvalue problems, combining it with a method developed by one of the authors (Jpn. J. Indust. Appl. Math. 16 (1998) 307).     

A formula for the 2-norm distance from a matrix to the set of matrices with multiple eigenvalues

Summary. We prove that the 2-norm distance from an matrix A to the matrices that have a multiple eigenvalue is equal to where the singular values are ordered nonincreasingly. Therefore, the 2-norm distance from A to the set of matrices with multiple eigenvalues is Received February 19, 1998 / Revised version received July 15, 1998 / Published online: July 7, 1999     

On close eigenvalues of tridiagonal matrices

Summary. A symmetric tridiagonal matrix with a multiple eigenvalue must have a zero subdiagonal element and must be a direct sum of two complementary blocks, both of which have the eigenvalue. Yet it is well known that a small spectral gap does not necessarily imply that some is small, as is demonstrated by the Wilkinson matrix. In this note, it is shown that a pair of close eigenvalues can only arise from two complementary blocks on the diagonal, in spite of the fact that the coupling the two blocks may not be small. In particular, some explanatory bounds are derived and a connection to the Lanczos algorithm is observed. The nonsymmetric problem is also included. Received April 8, 1992 / Revised version received September 21, 1994     

Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices

The sum of the largest eigenvalues of a symmetric matrix is a nonsmooth convex function of the matrix elements. Max characterizations for this sum are established, giving a concise characterization of the subdifferential in terms of a dual matrix. This leads to a very useful characterization of the generalized gradient of the following convex composite function: the sum of the largest eigenvalues of a smooth symmetric matrix-valued function of a set of real parameters. The dual matrix provides the information required to either verify first-order optimality conditions at a point or to generate a descent direction for the eigenvalue sum from that point, splitting a multiple eigenvalue if necessary. Connections with the classical literature on sums of eigenvalues and eigenvalue perturbation theory are discussed. Sums of the largest eigenvalues in the absolute value sense are also addressed.This paper is dedicated to Phil Wolfe on the occasion of his 65th birthday.The work of this author was supported by the National Science Foundation under grants CCR-8802408 and CCR-9101640.The work of this author was supported in part during a visit to Argonne National Laboratory by the Applied Mathematical Sciences subprogram of the Office of Energy Research of the U.S. Department of Energy under contract W-31-109-Eng-38, and in part during a visit to the Courant Institute by the U.S. Department of Energy under Contract DEFG0288ER25053.     

A global harmonic Arnoldi method for large non-Hermitian eigenproblems with an application to multiple eigenvalue problems

The global Arnoldi method can be used to compute exterior eigenpairs of a large non-Hermitian matrix A, but it does not work well for interior eigenvalue problems. Based on the global Arnoldi process that generates an F-orthonormal basis of a matrix Krylov subspace, we propose a global harmonic Arnoldi method for computing certain harmonic F-Ritz pairs that are used to approximate some interior eigenpairs. We propose computing the F-Rayleigh quotients of the large non-Hermitian matrix with respect to harmonic F-Ritz vectors and taking them as new approximate eigenvalues. They are better and more reliable than the harmonic F-Ritz values. The global harmonic Arnoldi method inherits convergence properties of the harmonic Arnoldi method applied to a larger matrix whose distinct eigenvalues are the same as those of the original given matrix. Some properties of the harmonic F-Ritz vectors are presented. As an application, assuming that A is diagonalizable, we show that the global harmonic Arnoldi method is able to solve multiple eigenvalue problems both in theory and in practice. To be practical, we develop an implicitly restarted global harmonic Arnoldi algorithm with certain harmonic F-shifts suggested. In particular, this algorithm can be adaptively used to solve multiple eigenvalue problems. Numerical experiments show that the algorithm is efficient for the eigenproblem and is reliable for quite ill-conditioned multiple eigenproblems.     

Eigenvalue perturbation bounds for Hermitian block tridiagonal matrices

We derive new perturbation bounds for eigenvalues of Hermitian matrices with block tridiagonal structure. The main message of this paper is that an eigenvalue is insensitive to blockwise perturbation, if it is well-separated from the spectrum of the diagonal blocks nearby the perturbed blocks. Our bound is particularly effective when the matrix is block-diagonally dominant and graded. Our approach is to obtain eigenvalue bounds via bounding eigenvector components, which is based on the observation that an eigenvalue is insensitive to componentwise perturbation if the corresponding eigenvector components are small. We use the same idea to explain two well-known phenomena, one concerning aggressive early deflation used in the symmetric tridiagonal QR algorithm and the other concerning the extremal eigenvalues of Wilkinson matrices.     

Interlacing properties of the eigenvalues of some matrix classes

We establish the eigenvalue interlacing property (i.e. the smallest real eigenvalue of a matrix is less than the smallest real eigenvalue of any of its principal submatrices) for the class of matrices introduced by Kotelyansky (all principal and almost principal minors of these matrices are positive). We show that certain generalizations of Kotelyansky and totally positive matrices possess this property. We also prove some interlacing inequalities for the other eigenvalues of Kotelyansky matrices.     

On graphs with exactly one anti-adjacency eigenvalue and beyond

The anti-adjacency matrix of a graph is constructed from the distance matrix of a graph by keeping each row and each column only the largest distances. This matrix can be interpreted as the opposite of the adjacency matrix, which is instead constructed from the distance matrix of a graph by keeping in each row and each column only the distances equal to 1. The (anti-)adjacency eigenvalues of a graph are those of its (anti-)adjacency matrix. Employing a novel technique introduced by Haemers (2019) [9], we characterize all connected graphs with exactly one positive anti-adjacency eigenvalue, which is an analog of Smith's classical result that a connected graph has exactly one positive adjacency eigenvalue iff it is a complete multipartite graph. On this basis, we identify the connected graphs with all but at most two anti-adjacency eigenvalues equal to ?2 and 0. Moreover, for the anti-adjacency matrix we determine the HL-index of graphs with exactly one positive anti-adjacency eigenvalue, where the HL-index measures how large in absolute value may be the median eigenvalues of a graph. We finally propose some problems for further study.     

The Clarke and Michel-Penot Subdifferentials of the Eigenvalues of a Symmetric Matrix

We calculate the Clarke and Michel-Penot subdifferentials of the function which maps a symmetric matrix to its mth largest eigenvalue. We show these two subdifferentials coincide, and are identical for all choices of index m corresponding to equal eigenvalues. Our approach is via the generalized directional derivatives of the eigenvalue function, thereby completing earlier studies on the classical directional derivative.     

On condition numbers of polynomial eigenvalue problems

In this paper, we investigate condition numbers of eigenvalue problems of matrix polynomials with nonsingular leading coefficients, generalizing classical results of matrix perturbation theory. We provide a relation between the condition numbers of eigenvalues and the pseudospectral growth rate. We obtain that if a simple eigenvalue of a matrix polynomial is ill-conditioned in some respects, then it is close to be multiple, and we construct an upper bound for this distance (measured in the euclidean norm). We also derive a new expression for the condition number of a simple eigenvalue, which does not involve eigenvectors. Moreover, an Elsner-like perturbation bound for matrix polynomials is presented.     

Sensitivity Analysis of Multiple Eigenvalues (Ⅱ)

This paper discusses the sensitivity of semisimple multiple eigenvalues and corresponding invariant subspaces of a complex (or real) $n\times n$ matrix analytically dependent on several parameters. Some results of this paper may be useful for investigating robust multiple eigenvalue assignment in control system design.