A geometric theory for preconditioned inverse iteration I: Extrema of the Rayleigh quotient

The discretization of eigenvalue problems for partial differential operators is a major source of matrix eigenvalue problems having very large dimensions, but only some of the smallest eigenvalues together with the eigenvectors are to be determined. Preconditioned inverse iteration (a “matrix-free” method) derives from the well-known inverse iteration procedure in such a way that the associated system of linear equations is solved approximately by using a (multigrid) preconditioner. A new convergence analysis for preconditioned inverse iteration is presented. The preconditioner is assumed to satisfy some bound for the spectral radius of the error propagation matrix resulting in a simple geometric setup. In this first part the case of poorest convergence depending on the choice of the preconditioner is analyzed. In the second part the dependence on all initial vectors having a fixed Rayleigh quotient is considered. The given theory provides sharp convergence estimates for the eigenvalue approximations showing that multigrid eigenvalue/vector computations can be done with comparable efficiency as known from multigrid methods for boundary value problems.     

On optimizing the sum of the Rayleigh quotient and the generalized Rayleigh quotient on the unit sphere

Given symmetric matrices B,D∈? ^n×n and a symmetric positive definite matrix W∈? ^n×n , maximizing the sum of the Rayleigh quotient x ^{??/sup> D x and the generalized Rayleigh quotient $\frac{\mathbf{x}^{\top}B \mathbf{x}}{\vphantom{\mathrm{I}^{\mathrm{I}}}\mathbf{x}^{\top}W\mathbf{x}}$ on the unit sphere not only is of mathematical interest in its own right, but also finds applications in practice. In this paper, we first present a real world application arising from the sparse Fisher discriminant analysis. To tackle this problem, our first effort is to characterize the local and global maxima by investigating the optimality conditions. Our results reveal that finding the global solution is closely related with a special extreme nonlinear eigenvalue problem, and in the special case D=μW?(μ>0), the set of the global solutions is essentially an eigenspace corresponding to the largest eigenvalue of a specially-defined matrix. The characterization of the global solution not only sheds some lights on the maximization problem, but motives a starting point strategy to obtain the global maximizer for any monotonically convergent iteration. Our second part then realizes the Riemannian trust-region method of Absil, Baker and Gallivan (Found. Comput. Math. 7:303??30, 2007) into a practical algorithm to solve this problem, which enjoys the nice convergence properties: global convergence and local superlinear convergence. Preliminary numerical tests are carried out and empirical evaluation of its performance is reported.}     

On convergence of the inexact Rayleigh quotient iteration with MINRES

For the Hermitian inexact Rayleigh quotient iteration (RQI), we present a new general theory, independent of iterative solvers for shifted inner linear systems. The theory shows that the method converges at least quadratically under a new condition, called the uniform positiveness condition, that may allow the residual norm _ξk≥1${\xi }_k\ge 1$ of the inner linear system at outer iteration k+1$k+1$ and can be considerably weaker than the condition _ξk≤ξ<1${\xi }_k\le \xi <1$ with ξ$\xi$ a constant not near one commonly used in the literature. We consider the convergence of the inexact RQI with the unpreconditioned and tuned preconditioned MINRES methods for the linear systems. Some attractive properties are derived for the residuals obtained by MINRES. Based on them and the new general theory, we make a refined analysis and establish a number of new convergence results. Let ‖_rk‖$\Vert r_k\Vert$ be the residual norm of approximating eigenpair at outer iteration k$k$. Then all the available cubic and quadratic convergence results require _ξk=O(‖_rk‖)${\xi }_k=O\left(\Vert r_k\Vert \right)$ and _ξk≤ξ${\xi }_k\le \xi$ with a fixed ξ$\xi$ not near one, respectively. Fundamentally different from these, we prove that the inexact RQI with MINRES generally converges cubically, quadratically and linearly provided that _ξk≤ξ${\xi }_k\le \xi$ with a constant ξ<1$\xi <1$ not near one, _ξk=1−O(‖_rk‖)${\xi }_k=1-O\left(\Vert r_k\Vert \right)$ and _ξk=1−O(‖_rk‖²)${\xi }_k=1-O\left({\Vert r_k\Vert }^2\right)$, respectively. The new convergence conditions are much more relaxed than ever before. The theory can be used to design practical stopping criteria to implement the method more effectively. Numerical experiments confirm our results.     

Generalized Rayleigh quotient and finite element two-grid discretization schemes

This study discusses generalized Rayleigh quotient and high efficiency finite element discretization schemes. Some results are as follows: 1) Rayleigh quotient accelerate technique is extended to nonselfadjoint problems. Generalized Rayleigh quotients of operator form and weak form are defined and the basic relationship between approximate eigenfunction and its generalized Rayleigh quotient is established. 2) New error estimates are obtained by replacing the ascent of exact eigenvalue with the ascent of fin...     

Iterative minimization of the Rayleigh quotient by block steepest descent iterations

The topic of this paper is the convergence analysis of subspace gradient iterations for the simultaneous computation of a few of the smallest eigenvalues plus eigenvectors of a symmetric and positive definite matrix pair (A,M). The methods are based on subspace iterations for A ^{? 1}M and use the Rayleigh‐Ritz procedure for convergence acceleration. New sharp convergence estimates are proved by generalizing estimates, which have been presented for vectorial steepest descent iterations (see SIAM J. Matrix Anal. Appl., 32(2):443‐456, 2011). Copyright © 2013 John Wiley & Sons, Ltd.     

A generalized Rayleigh quotient for eigenvalue problems nonlinear in the parameter

A functional is given which generalizes the Rayleigh quotient to eigenvalue problems for linear operators where the eigenvalue parameter appears nonlinearly. Particular emphasis is given to the development of perturbation-type results for eigenvalues and characteristic values which generalize the classical results. Applications are made to eigenvalue and characteristic value problems for integral and matrix operators and to the critical length problem for integral operators. Both symmetric and nonsymmetric operators are treated.The author would like to acknowledge the work of Gloria Golberg in the preparation of this paper.     

A skew normal dilation on the numerical range of an operator

Simple facts about the Poincaré-Neumann double layer potential are used in the construction of a normal dilation, on the numerical range of an arbitrary Hilbert space operator. Recent and old ideas in the theory of the numerical range are unified by this framework. A couple of mapping results for the numerical range are derived.Revised version: 15 July 2002First author partially supported by the National Science Foundation Grant DMS 0100367, Second author supported by STINT, The Swedish Foundation for International Cooperation in Research and Higher Education.     

Extrema Constrained by a Family of Curves and Local Extrema

This paper considers the connections between the local extrema of a function f:DR and the local extrema of the restrictions of f to specific subsets of D. In particular, such subsets may be parametrized curves, integral manifolds of a Pfaff system, Pfaff inequations. The paper shows the existence of C ¹ or C ²-curves containing a given sequence of points. Such curves are then exploited to establish the connections between the local extrema of f and the local extrema of f constrained by the family of C ¹ or C ²-curves. Surprisingly, what is true for C ¹-curves fails to be true in part for C ²-curves. Sufficient conditions are given for a point to be a global minimum point of a convex function with respect to a family of curves.     

Tuned preconditioners for inexact two‐sided inverse and Rayleigh quotient iteration

Convergence results are provided for inexact two‐sided inverse and Rayleigh quotient iteration, which extend the previously established results to the generalized non‐Hermitian eigenproblem and inexact solves with a decreasing solve tolerance. Moreover, the simultaneous solution of the forward and adjoint problem arising in two‐sided methods is considered, and the successful tuning strategy for preconditioners is extended to two‐sided methods, creating a novel way of preconditioning two‐sided algorithms. Furthermore, it is shown that inexact two‐sided Rayleigh quotient iteration and the inexact two‐sided Jacobi‐Davidson method (without subspace expansion) applied to the generalized preconditioned eigenvalue problem are equivalent when a certain number of steps of a Petrov–Galerkin–Krylov method is used and when this specific tuning strategy is applied. Copyright © 2014 John Wiley & Sons, Ltd.     

A method based on Rayleigh quotient gradient flow for extreme and interior eigenvalue problems

Recently, a continuous method has been proposed by Golub and Liao as an alternative way to solve the minimum and interior eigenvalue problems. According to their numerical results, their method seems promising. This article is an extension along this line. In this article, firstly, we convert an eigenvalue problem to an equivalent constrained optimization problem. Secondly, using the Karush-Kuhn-Tucker conditions of this equivalent optimization problem, we obtain a variant of the Rayleigh quotient gradient flow, which is formulated by a system of differential-algebraic equations. Thirdly, based on the Rayleigh quotient gradient flow, we give a practical numerical method for the minimum and interior eigenvalue problems. Finally, we also give some numerical experiments of our method, the Golub and Liao method, and EIGS (a Matlab implementation for computing eigenvalues using restarted Arnoldi’s method) for some typical eigenvalue problems. Our numerical experiments indicate that our method seems promising for most test problems.     

Similarity of an operator to an isometric operator

Leningrad State University. Translated from Funktsional'nyi Analiz i Ego Prilozheniya, Vol. 23, No. 2, pp. 77–78, April–June, 1989.     

Duality and normal parts of operator modules

For an operator bimodule X over von Neumann algebras A⊆B(H) and B⊆B(K), the space of all completely bounded A,B-bimodule maps from X into B(K,H), is the bimodule dual of X. Basic duality theory is developed with a particular attention to the Haagerup tensor product over von Neumann algebras. To X a normal operator bimodule X_n is associated so that completely bounded A,B-bimodule maps from X into normal operator bimodules factorize uniquely through X_n. A construction of X_n in terms of biduals of X, A and B is presented. Various operator bimodule structures are considered on a Banach bimodule admitting a normal such structure.     

一般欧氏空间上的广义规范算子与广义规范矩阵

研究了一般欧氏空间上的广义规范算子与广义规范矩阵的性质。