Fast Structured Matrix Computations: Tensor Rank and Cohn–Umans Method

We discuss a generalization of the Cohn–Umans method, a potent technique developed for studying the bilinear complexity of matrix multiplication by embedding matrices into an appropriate group algebra. We investigate how the Cohn–Umans method may be used for bilinear operations other than matrix multiplication, with algebras other than group algebras, and we relate it to Strassen’s tensor rank approach, the traditional framework for investigating bilinear complexity. To demonstrate the utility of the generalized method, we apply it to find the fastest algorithms for forming structured matrix–vector product, the basic operation underlying iterative algorithms for structured matrices. The structures we study include Toeplitz, Hankel, circulant, symmetric, skew-symmetric, f-circulant, block Toeplitz–Toeplitz block, triangular Toeplitz matrices, Toeplitz-plus-Hankel, sparse/banded/triangular. Except for the case of skew-symmetric matrices, for which we have only upper bounds, the algorithms derived using the generalized Cohn–Umans method in all other instances are the fastest possible in the sense of having minimum bilinear complexity. We also apply this framework to a few other bilinear operations including matrix–matrix, commutator, simultaneous matrix products, and briefly discuss the relation between tensor nuclear norm and numerical stability.     

Fast Algorithms with Preprocessing for Matrix-Vector Multiplication Problems

In this paper the problem of complexity of multiplication of a matrix with a vector is studied for Toeplitz, Hankel, Vandermonde, and Cauchy matrices and for matrices connected with them (i.e., for transpose, inverse, and transpose to inverse matrices). The proposed algorithms have complexities of at most O(n log²n) flops and in a number of cases they improve the known estimates. In these algorithms, in a separate preprocessing phase, are singled out all the actions on the preparation of a given matrix which aimed at the reduction of the complexity of the second stage of computations directly connected with multiplication by an arbitrary vector. Effective algorithms for computing the Vandermonde determinant and the determination of a Cauchy matrix are given.     

Minimal condition number for positive definite Hankel matrices using semidefinite programming

We present a semidefinite programming approach for computing optimally conditioned positive definite Hankel matrices of order n. Unlike previous approaches, our method is guaranteed to find an optimally conditioned positive definite Hankel matrix within any desired tolerance. Since the condition number of such matrices grows exponentially with n, this is a very good test problem for checking the numerical accuracy of semidefinite programming solvers. Our tests show that semidefinite programming solvers using fixed double precision arithmetic are not able to solve problems with n>30. Moreover, the accuracy of the results for 24?n?30 is questionable. In order to accurately compute minimal condition number positive definite Hankel matrices of higher order, we use a Mathematica 6.0 implementation of the SDPHA solver that performs the numerical calculations in arbitrary precision arithmetic. By using this code, we have validated the results obtained by standard codes for n?24, and we have found optimally conditioned positive definite Hankel matrices up to n=100.     

Structured FISTA for image restoration

In this paper, we propose an efficient numerical scheme for solving some large‐scale ill‐posed linear inverse problems arising from image restoration. In order to accelerate the computation, two different hidden structures are exploited. First, the coefficient matrix is approximated as the sum of a small number of Kronecker products. This procedure not only introduces one more level of parallelism into the computation but also enables the usage of computationally intensive matrix–matrix multiplications in the subsequent optimization procedure. We then derive the corresponding Tikhonov regularized minimization model and extend the fast iterative shrinkage‐thresholding algorithm (FISTA) to solve the resulting optimization problem. Because the matrices appearing in the Kronecker product approximation are all structured matrices (Toeplitz, Hankel, etc.), we can further exploit their fast matrix–vector multiplication algorithms at each iteration. The proposed algorithm is thus called structured FISTA (sFISTA). In particular, we show that the approximation error introduced by sFISTA is well under control and sFISTA can reach the same image restoration accuracy level as FISTA. Finally, both the theoretical complexity analysis and some numerical results are provided to demonstrate the efficiency of sFISTA.     

A fast method to block-diagonalize a Hankel matrix

In this paper, we consider an approximate block diagonalization algorithm of an n×n real Hankel matrix in which the successive transformation matrices are upper triangular Toeplitz matrices, and propose a new fast approach to compute the factorization in O(n ²) operations. This method consists on using the revised Bini method (Lin et al., Theor Comp Sci 315: 511–523, 2004). To motivate our approach, we also propose an approximate factorization variant of the customary fast method based on Schur complementation adapted to the n×n real Hankel matrix. All algorithms have been implemented in Matlab and numerical results are included to illustrate the effectiveness of our approach.     

Analysis of the Structured Total Least Squares Problem for Hankel/Toeplitz Matrices

The Structured Total Least Squares (STLS) problem is a natural extension of the Total Least Squares (TLS) approach when structured matrices are involved and a similarly structured rank deficient approximation of that matrix is desired. In many of those cases the STLS approach yields a Maximum Likelihood (ML) estimate as opposed to, e.g., TLS.In this paper we analyze the STLS problem for Hankel matrices (the theory can be extended in a straightforward way to Toeplitz matrices, block Hankel and block Toeplitz matrices). Using a particular parametrisation of rank-deficient Hankel matrices, we show that this STLS problem suffers from multiple local minima, the properties of which depend on the parameters of the new parametrisation. The latter observation makes initial estimates an important issue in STLS problems and a new initialization method is proposed. The new initialization method is applied to a speech compression example and the results confirm the improved performance compared to other previously proposed initialization methods.     

High‐performance direct algorithms for computing the sign function of triangular matrices

Algorithms and implementations for computing the sign function of a triangular matrix are fundamental building blocks for computing the sign of arbitrary square real or complex matrices. We present novel recursive and cache‐efficient algorithms that are based on Higham's stabilized specialization of Parlett's substitution algorithm for computing the sign of a triangular matrix. We show that the new recursive algorithms are asymptotically optimal in terms of the number of cache misses that they generate. One algorithm that we present performs more arithmetic than the nonrecursive version, but this allows it to benefit from calling highly optimized matrix multiplication routines; the other performs the same number of operations as the nonrecursive version, suing custom computational kernels instead. We present implementations of both, as well as a cache‐efficient implementation of a block version of Parlett's algorithm. Our experiments demonstrate that the blocked and recursive versions are much faster than the previous algorithms and that the inertia strongly influences their relative performance, as predicted by our analysis.     

满秩Hankel矩阵的拟直分解

本文研究行满秩Hankel矩阵分解为一个真正的(proper)Hankel矩阵与一个退化的(de- generate)Hankel矩阵之拟直和的存在性及唯一性问题.     

On two particular cases of solving the normal Hankel problem

The normal Hankel problem is one of characterizing all the complex matrices that are normal and Hankel at the same time. The matrix classes that can contain normal Hankel matrices admit a parameterization by real 2 × 2 matrices with determinant one. Here, the normal Hankel problem is solved in the case where the characteristic matrix of a given class is an order two Jordan block for the eigenvalue 1 or ?1.     

On Distinguished Solutions of Truncated Matricial Hamburger Moment Problems

We study two slightly different versions of the truncated matricial Hamburger moment problem. A central topic is the construction and investigation of distinguished solutions of both moment problems under consideration. These solutions turn out to be nonnegative Hermitian q × q Borel measures on the real axis which are concentrated on a finite number of points. These points and the corresponding masses will be explicitly described in terms of the given data. Furthermore, we investigate a particular class of sequences (s_j)^∞_{j = 0} of complex q × q matrices for which the corresponding infinite matricial Hamburger moment problem has a unique solution. Our approach is mainly algebraic. It is based on the use of particular matrix polynomials constructed from a nonnegative Hermitian block Hankel matrix. These matrix polynomials are immediate generalizations of the monic orthogonal matrix polynomials associated with a positive Hermitian block Hankel matrix. We generalize a classical theorem due to Kronecker on infinite Hankel matrices of finite rank to block Hankel matrices and discuss its consequences for the nonnegative Hermitian case.     

Tangential Nevanlinna-Pick Interpolation and Its Connection with Hamburger Matrix Moment Problem

This paper is concerned with the solution of a certain tangential Nevanlinna-Pick interpolation for Nevanlinna functions. We use the so-called block Hankel vector method to establish two intrinsic connections between the tangential Nevanlinna-Pick interpolation in the Nevanlinna class and the truncated Hamburger matrix moment problem associated with the block Hankel vector under consideration: one is a congruent relationship between their information matrices, and the other is a divisor-remainder connection between their solutions. These investigations generalize our previous work on the Nevanlinna-Pick interpolation and power matrix moment problem.     

On the numerical condition of a generalized Hankel eigenvalue problem

The generalized eigenvalue problem with H a Hankel matrix and the corresponding shifted Hankel matrix occurs in number of applications such as the reconstruction of the shape of a polygon from its moments, the determination of abscissa of quadrature formulas, of poles of Padé approximants, or of the unknown powers of a sparse black box polynomial in computer algebra. In many of these applications, the entries of the Hankel matrix are only known up to a certain precision. We study the sensitivity of the nonlinear application mapping the vector of Hankel entries to its generalized eigenvalues. A basic tool in this study is a result on the condition number of Vandermonde matrices with not necessarily real abscissas which are possibly row-scaled. B. Beckermann was supported in part by INTAS research network NaCCA 03-51-6637. G. H. Golub was supported in part by DOE grant DE-FC-02-01ER41177. G. Labahn was supported in part by NSERC and MITACS Canada grants.     

Hierarchical Schur complement preconditioner for the stochastic Galerkin finite element methods

Use of the stochastic Galerkin finite element methods leads to large systems of linear equations obtained by the discretization of tensor product solution spaces along their spatial and stochastic dimensions. These systems are typically solved iteratively by a Krylov subspace method. We propose a preconditioner, which takes an advantage of the recursive hierarchy in the structure of the global matrices. In particular, the matrices posses a recursive hierarchical two‐by‐two structure, with one of the submatrices block diagonal. Each of the diagonal blocks in this submatrix is closely related to the deterministic mean‐value problem, and the action of its inverse is in the implementation approximated by inner loops of Krylov iterations. Thus, our hierarchical Schur complement preconditioner combines, on each level in the approximation of the hierarchical structure of the global matrix, the idea of Schur complement with loops for a number of mutually independent inner Krylov iterations, and several matrix–vector multiplications for the off‐diagonal blocks. Neither the global matrix nor the matrix of the preconditioner need to be formed explicitly. The ingredients include only the number of stiffness matrices from the truncated Karhunen–Loève expansion and a good preconditioned for the mean‐value deterministic problem. We provide a condition number bound for a model elliptic problem, and the performance of the method is illustrated by numerical experiments. Copyright © 2013 John Wiley & Sons, Ltd.     

由特征值唯一确定的实3×3Hankel矩阵(英文)

本文研究了由特征值唯一确定的3×3实Hankel矩阵.借助于M.Fielder[1]的结论并经过细致的讨论,得到3×3实Hankel矩阵由其特征值唯一确定的充分必要条件,刻画了3×3实Hankel矩阵的一种特征值性质.     

Relaxation strategies for nested Krylov methods

This paper studies computational aspects of Krylov methods for solving linear systems where the matrix–vector products dominate the cost of the solution process because they have to be computed via an expensive approximation procedure. In recent years, so-called relaxation strategies for tuning the precision of the matrix–vector multiplications in Krylov methods have proved to be effective for a range of problems. In this paper, we will argue that the gain obtained from such strategies is often limited. Another important strategy for reducing the work in the matrix–vector products is preconditioning the Krylov method by another iterative Krylov method. Flexible Krylov methods are Krylov methods designed for this situation. We combine these two approaches for reducing the work in the matrix–vector products. Specifically, we present strategies for choosing the precision of the matrix–vector products in several flexible Krylov methods as well as for choosing the accuracy of the variable preconditioner such that the overall method is as efficient as possible. We will illustrate this computational scheme with a Schur-complement system that arises in the modeling of global ocean circulation.     

Generalized Hankel matrices of Markov parameters and their applications to control problems

The concept of Hankel matrices of Markov parameters associated with two polynomials is generalized for matrices. The generalized Hankel matrices of Markov parameters are then used to develop methods for testing the relative primeness of two matrices A and B, for determining stability and inertia of a matrix, and for constructing a class of matrices C such that A + C has a desired spectrum. Neither the method of construction of the generalized Hankel matrices nor the methods developed using these matrices require explicit computation of the characteristic polynomial of A (or of B).     

Multivariate Polynomials,Duality, and Structured Matrices

We first review the basic properties of the well known classes of Toeplitz, Hankel, Vandermonde, and other related structured matrices and reexamine their correlation to operations with univariate polynomials. Then we define some natural extensions of such classes of matrices based on their correlation to multivariate polynomials. We describe the correlation in terms of the associated operators of multiplication in the polynomial ring and its dual space, which allows us to generalize these structures to the multivariate case. Multivariate Toeplitz, Hankel, and Vandermonde matrices, Bezoutians, algebraic residues, and relations between them are studied. Finally, we show some applications of this study to rootfinding problems for a system of multivariate polynomial equations, where the dual space, algebraic residues, Bezoutians, and other structured matrices play an important role. The developed techniques enable us to obtain a better insight into the major problems of multivariate polynomial computations and to improve substantially the known techniques of the study of these problems. In particular, we simplify and/or generalize the known reduction of the multivariate polynomial systems to the matrix eigenproblem, the derivation of the Bézout and Bernshtein bounds on the number of the roots, and the construction of multiplication tables. From the algorithmic and computational complexity point, we yield acceleration by one order of magnitude of the known methods for some fundamental problems of solving multivariate polynomial systems of equations.     

On a Sparse Representation of an n-Dimensional Laplacian in Wavelet Coordinates

Important parts of adaptive wavelet methods are well-conditioned wavelet stiffness matrices and an efficient approximate multiplication of quasi-sparse stiffness matrices with vectors in wavelet coordinates. Therefore it is useful to develop a well-conditioned wavelet basis with respect to which both the mass and stiffness matrices are sparse in the sense that the number of nonzero elements in each column is bounded by a constant. Consequently, the stiffness matrix corresponding to the n-dimensional Laplacian in the tensor product wavelet basis is also sparse. Then a matrix–vector multiplication can be performed exactly with linear complexity. In this paper, we construct a wavelet basis based on Hermite cubic splines with respect to which both the mass matrix and the stiffness matrix corresponding to a one-dimensional Poisson equation are sparse. Moreover, a proposed basis is well-conditioned on low decomposition levels. Small condition numbers for low decomposition levels and a sparse structure of stiffness matrices are kept for any well-conditioned second order partial differential equations with constant coefficients; furthermore, they are independent of the space dimension.     

Sufficient conditions for permutation equivalence to a WHS-matrix

Square matrices with positive leading principal minors, called WHS-matrices (weak Hawkins–Simon), are considered in economics. Some sufficient conditions for a matrix to be a WHS-matrix after suitable row and/or column permutations have recently appeared in the literature. New and unified proofs and generalizations of some results to rectangular matrices are given. In particular, it is shown that if left multiplication of a rectangular matrix A by some nonnegative matrix is upper triangular with positive diagonal, then some row pemutation of A is a WHS-matrix. For a nonsingular A with either the first nonzero entry of each of its rows positive or the last nonzero entry of each column of A ^?1 positive, again some row permutation of A is a WHS-matrix. In addition, any rectangular full rank semipositive matrix is shown to be permutation equivalent to a WHS-matrix.