A comparison between the complex symmetric based and classical computation of the singular value decomposition of normal matrices

An algorithm for computing the singular value decomposition of normal matrices using intermediate complex symmetric matrices is proposed. This algorithm, as most eigenvalue and singular value algorithms, consists of two steps. It is based on combining the unitarily equivalence of normal matrices to complex symmetric tridiagonal form with the symmetric singular value decomposition of complex symmetric matrices. Numerical experiments are included comparing several algorithms, with respect to speed and accuracy, for computing the symmetric singular value decomposition (also known as the Takagi factorization). Next we compare the novel approach with the classical Golub-Kahan method for computing the singular value decomposition of normal matrices: it is faster, consumes less memory, but on the other hand the results are significantly less accurate.     

Bounds for the extremal eigenvalues of a class of symmetric tridiagonal matrices with applications

We consider a class of symmetric tridiagonal matrices which may be viewed as perturbations of Toeplitz matrices. The Toeplitz structure is destroyed since two elements on each off-diagonal are perturbed. Based on a careful analysis, we derive sharp bounds for the extremal eigenvalues of this class of matrices in terms of the original data of the given matrix. In this way, we also obtain a lower bound for the smallest singular value of certain matrices. Some numerical results indicate that our bounds are extremely good.     

Exploiting Mixed Precision for Computing Eigenvalues of Symmetric Matrices and Singular Values

The use of higher precision preconditioning for the symmetric eigenvalue problem and the singular value problem of general non–structured non–graded matrices are discussed. The matrix Q from the QR–decomposition as a preconditioner, applied to A with higher precision, in combination with Jacobi's method seems to allow the computation of all eigenvalues of symmetric positive definite matrices rsp. all singular values of general matrices to nearly full accuracy. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

A note on properties of splittings of singular symmetric positive semidefinite matrices

Summary. Recently, Benzi and Szyld have published an important paper [1] concerning the existence and uniqueness of splittings for singular matrices. However, the assertion in Theorem 3.9 on the inheriting property of P-regular splitting for singular symmetric positive semidefinite matrices seems to be incorrect. As a complement of paper [1], in this short note we point out that if a matrix T is resulted from a P-regular splitting of a symmetric positive semidefinite matrix A, then splittings induced by T are not all P-regular. Received January 7, 1999 / Published online December 19, 2000     

On tridiagonal matrices unitarily equivalent to normal matrices

In this article the unitary equivalence transformation of normal matrices to tridiagonal form is studied.It is well-known that any matrix is unitarily equivalent to a tridiagonal matrix. In case of a normal matrix the resulting tridiagonal inherits a strong relation between its super- and subdiagonal elements. The corresponding elements of the super- and subdiagonal will have the same absolute value.In this article some basic facts about a unitary equivalence transformation of an arbitrary matrix to tridiagonal form are firstly studied. Both an iterative reduction based on Krylov sequences as a direct tridiagonalization procedure via Householder transformations are reconsidered. This equivalence transformation is then applied to the normal case and equality of the absolute value between the super- and subdiagonals is proved. Self-adjointness of the resulting tridiagonal matrix with regard to a specific scalar product is proved. Properties when applying the reduction on symmetric, skew-symmetric, Hermitian, skew-Hermitian and unitary matrices and their relations with, e.g., complex symmetric and pseudo-symmetric matrices are presented.It is shown that the reduction can then be used to compute the singular value decomposition of normal matrices making use of the Takagi factorization. Finally some extra properties of the reduction as well as an efficient method for computing a unitary complex symmetric decomposition of a normal matrix are given.     

Lanczos-type Methods for Continuation Problems

We study the Lanczos type methods for continuation problems. First we indicate how the symmetric Lanczos method may be used to solve both positive definite and indefinite linear systems. Furthermore, it can be used to monitor the simple bifurcation points on the solution curve of the eigenvalue problems. This includes computing the minimum eigenvalue, the minimum singular value, and the condition number of the partial tridiagonalizations of the coefficient matrices. The Ritz vector thus obtained can be applied to compute the tangent vector at the bifurcation point for branch-switching. Next, we indicate that the block or band Lanczos method can be used to monitor the multiple bifurcations as well as to solve the multiple right hand sides. We also show that the unsymmetric Lanczos method can be exploited to compute the minimum eigenvalue of a nearly symmetric matrix, and therefore to detect the simple bifurcation point as well. Some preconditioning techniques are discussed. Sample numerical results are reported. Our test problems include second order semilinear elliptic eigenvalue problems. © 1997 by John Wiley & Sons, Ltd.     

Direct and Inverse Eigenvalue Problems for Diagonal-Plus-Semiseparable Matrices

In this paper we study both direct and inverse eigenvalue problems for diagonal-plus-semiseparable (dpss) matrices. In particular, we show that the computation of the eigenvalues of a symmetric dpss matrix can be reduced by a congruence transformation to solving a generalized symmetric definite tridiagonal eigenproblem. Using this reduction, we devise a set of recurrence relations for evaluating the characteristic polynomial of a dpss matrix in a stable way at a linear time. This in turn allows us to apply divide-and-conquer eigenvalue solvers based on functional iterations directly to dpss matrices without performing any preliminary reduction into a tridiagonal form. In the second part of the paper, we exploit the structural properties of dpss matrices to solve the inverse eigenvalue problem of reconstructing a symmetric dpss matrix from its spectrum and some other informations. Finally, applications of our results to the computation of a QR factorization of a Cauchy matrix with real nodes are provided.     

Singular random matrix decompositions: distributions

Assuming that Y has a singular matrix variate elliptically contoured distribution with respect to the Hausdorff measure, the distributions of several matrices associated to QR, modified QR, SV and polar decompositions of matrix Y are determined, for central and non-central, non-singular and singular cases, as well as their relationship to the Wishart and pseudo-Wishart generalized singular and non-singular distributions. Some of these results are also applied to two particular subfamilies of elliptical distributions, the singular matrix variate normal distribution and the singular matrix variate symmetric Pearson type VII distribution.     

Existence and uniqueness of splittings for stationary iterative methods with applications to alternating methods

Summary. Given a nonsingular matrix , and a matrix of the same order, under certain very mild conditions, there is a unique splitting , such that . Moreover, all properties of the splitting are derived directly from the iteration matrix . These results do not hold when the matrix is singular. In this case, given a matrix and a splitting such that , there are infinitely many other splittings corresponding to the same matrices and , and different splittings can have different properties. For instance, when is nonnegative, some of these splittings can be regular splittings, while others can be only weak splittings. Analogous results hold in the symmetric positive semidefinite case. Given a singular matrix , not for all iteration matrices there is a splitting corresponding to them. Necessary and sufficient conditions for the existence of such splittings are examined. As an illustration of the theory developed, the convergence of certain alternating iterations is analyzed. Different cases where the matrix is monotone, singular, and positive (semi)definite are studied. Received September 5, 1995 / Revised version received April 3, 1996     

Numerically Stable Generation of Correlation Matrices and Their Factors

Correlation matrices—symmetric positive semidefinite matrices with unit diagonal—are important in statistics and in numerical linear algebra. For simulation and testing it is desirable to be able to generate random correlation matrices with specified eigenvalues (which must be nonnegative and sum to the dimension of the matrix). A popular algorithm of Bendel and Mickey takes a matrix having the specified eigenvalues and uses a finite sequence of Givens rotations to introduce 1s on the diagonal. We give improved formulae for computing the rotations and prove that the resulting algorithm is numerically stable. We show by example that the formulae originally proposed, which are used in certain existing Fortran implementations, can lead to serious instability. We also show how to modify the algorithm to generate a rectangular matrix with columns of unit 2-norm. Such a matrix represents a correlation matrix in factored form, which can be preferable to representing the matrix itself, for example when the correlation matrix is nearly singular to working precision.     

Fast approximate truncated SVD

This paper presents a new method for the computation of truncated singular value decomposition (SVD) of an arbitrary matrix. The method can be qualified as deterministic because it does not use randomized schemes. The number of operations required is asymptotically lower than that using conventional methods for nonsymmetric matrices and is at a par with the best existing deterministic methods for unstructured symmetric ones. It slightly exceeds the asymptotical computational cost of SVD methods based on randomization; however, the error estimate for such methods is significantly higher than for the presented one. The method is one‐pass, that is, each value of the matrix is used just once. It is also readily parallelizable. In the case of full SVD decomposition, it is exact. In addition, it can be modified for a case when data are obtained sequentially rather than being available all at once. Numerical simulations confirm accuracy of the method.     

Algebraic multilevel iteration method for Stieltjes matrices

The numerical solution of elliptic selfadjoint second-order boundary value problems leads to a class of linear systems of equations with symmetric, positive definite, large and sparse matrices which can be solved iteratively using a preconditioned version of some algorithm. Such differential equations originate from various applications such as heat conducting and electromagnetics. Systems of equations of similar type can also arise in the finite element analysis of structures. We discuss a recursive method constructing preconditioners to a symmetric, positive definite matrix. An algebraic multilevel technique based on partitioning of the matrix in two by two matrix block form, approximating some of these by other matrices with more simple sparsity structure and using the corresponding Schur complement as a matrix on the lower level, is considered. The quality of the preconditioners is improved by special matrix polynomials which recursively connect the preconditioners on every two adjoining levels. Upper and lower bounds for the degree of the polynomials are derived as conditions for a computational complexity of optimal order for each level and for an optimal rate of convergence, respectively. The method is an extended and more accurate algebraic formulation of a method for nine-point and mixed five- and nine-point difference matrices, presented in some previous papers.     

On the Second-order Directional Derivatives of Singular Values of Matrices and Symmetric Matrix-valued Functions

The (parabolic) second-order directional derivatives of singular values of matrices and symmetric matrix-valued functions induced by real-valued functions play important roles in studying second-order optimality conditions for different types of matrix cone optimization problems. We propose a direct way to derive the formula for the second-order directional derivative of any eigenvalue of a symmetric matrix in Torki (Nonlinear Anal 46:1133–1150 2001), from which a formula for the second-order directional derivative of any singular value of a matrix is established. We demonstrate a formula for the second-order directional derivative of the symmetric matrix-valued function. As applications, the second-order derivative for the projection operator over the SDP cone is derived and used to get the second-order tangent set of the SDP cone in Bonnans and Shapiro (2000), and the tangent cone and the second-order tangent set of the epigraph of the nuclear norm are given as well.     

Weighted pseudoinverses and weighted normal pseudosolutions with singular weights

Weighted pseudoinverses with singular weights can be defined by a system of matrix equations. For one of such definitions, necessary and sufficient conditions are given for the corresponding system to have a unique solution. Representations of the pseudoinverses in terms of the characteristic polynomials of symmetrizable and symmetric matrices, as well as their expansions in matrix power series or power products, are obtained. A relationship is found between the weighted pseudoinverses and the weighted normal pseudosolutions, and iterative methods for calculating both pseudoinverses and pseudosolutions are constructed. The properties of the weighted pseudoinverses with singular weights are shown to extend the corresponding properties of weighted pseudoinverses with positive definite weights.     

An implicit QR algorithm for symmetric semiseparable matrices

The QR algorithm is one of the classical methods to compute the eigendecomposition of a matrix. If it is applied on a dense n × n matrix, this algorithm requires O(n³) operations per iteration step. To reduce this complexity for a symmetric matrix to O(n), the original matrix is first reduced to tridiagonal form using orthogonal similarity transformations. In the report (Report TW360, May 2003) a reduction from a symmetric matrix into a similar semiseparable one is described. In this paper a QR algorithm to compute the eigenvalues of semiseparable matrices is designed where each iteration step requires O(n) operations. Hence, combined with the reduction to semiseparable form, the eigenvalues of symmetric matrices can be computed via intermediate semiseparable matrices, instead of tridiagonal ones. The eigenvectors of the intermediate semiseparable matrix will be computed by applying inverse iteration to this matrix. This will be achieved by using an O(n) system solver, for semiseparable matrices. A combination of the previous steps leads to an algorithm for computing the eigenvalue decompositions of semiseparable matrices. Combined with the reduction of a symmetric matrix towards semiseparable form, this algorithm can also be used to calculate the eigenvalue decomposition of symmetric matrices. The presented algorithm has the same order of complexity as the tridiagonal approach, but has larger lower order terms. Numerical experiments illustrate the complexity and the numerical accuracy of the proposed method. Copyright © 2005 John Wiley & Sons, Ltd.     

Incomplete block-matrix factorization iterative methods for convection-diffusion problems

Standard Galerkin finite element methods or finite difference methods for singular perturbation problems lead to strongly unsymmetric matrices, which furthermore are in general notM-matrices. Accordingly, preconditioned iterative methods such as preconditioned (generalized) conjugate gradient methods, which have turned out to be very successful for symmetric and positive definite problems, can fail to converge or require an excessive number of iterations for singular perturbation problems.This is not so much due to the asymmetry, as it is to the fact that the spectrum can have both eigenvalues with positive and negative real parts, or eigenvalues with arbitrary small positive real parts and nonnegligible imaginary parts. This will be the case for a standard Galerkin method, unless the meshparameterh is chosen excessively small. There exist other discretization methods, however, for which the corresponding bilinear form is coercive, whence its finite element matrix has only eigenvalues with positive real parts; in fact, the real parts are positive uniformly in the singular perturbation parameter.In the present paper we examine the streamline diffusion finite element method in this respect. It is found that incomplete block-matrix factorization methods, both on classical form and on an inverse-free (vectorizable) form, coupled with a general least squares conjugate gradient method, can work exceptionally well on this type of problem. The number of iterations is sometimes significantly smaller than for the corresponding almost symmetric problem where the velocity field is close to zero or the singular perturbation parameter =1.The 2 ^nd author's research was sponsored by Control Data Corporation through its PACER fellowship program.The 3 ^rd author's research was supported by the Netherlands organization for scientific research (NWO).On leave from the Institute of Mathematics, Academy of Science, 1090 Sofia, P.O. Box 373, Bulgaria.     

A practical solution for KKT systems

We consider KKT systems of linear equations with a 2 × 2 block indefinite matrix whose (2, 2) block is zero. Such systems arise in many applications. Treating such matrices would encounter some intricacies, especially when its (1, 1) block, i.e., the stiffness matrix in term of computational mechanics, is rank-deficient. It is the rank-deficiency of the stiffness matrix that leads to the so-called rigid-displacement issue. This is believed to be one of the main reasons that many programmers would unwillingly give up the Lagrange multiplier method but select the penalty method. Based on the Sherman–Morrison formula and the conventional LDL^T decomposition for symmetric positive definite matrices, a robust direct solution is proposed, which is amenable to the conventional finite element codes, competent for both nonsingular and singular stiffness matrices, and particularly suitable to parallel computation. As a paradigm, the application to the element-free Galerkin method (EFGM) with the moving least squares interpolation is illustrated. Funded by the National Natural Science Foundation of China (NSFC), Project no. 90510019.     

Updating undamped structural models using displacement output feedback

A novel numerical method for updating stiffness matrix by displacement output feedback technique is presented. The required displacement output feedback gain matrix is determined by the QR-decompositions and the singular value decompositions of matrices, and thus the optimal approximation stiffness matrix is found and the large number of unmeasured and unknown eigeninformation of the original model is preserved. The proposed method is computationally efficient since it does not require iterations and the updated stiffness matrix is also symmetric and has the compact expression.     

Linear operators preserving certain functions on singular values of matrices

We study the structure of those linear operators on the rectangular complex or real matrix spaces that preserve certain functions on singular values. We first do a brief survey on the existing results in the area and then prove a theorem which covers and extends all of them. In particular. our theorem confirms two conjectures about the structure of those linear operators preserving the completely symmetric functions on powers of singular values of matrices.