The partial total least squares algorithm

In this paper, an improved algorithm PTLS for solving total least squares (TLS) problems AX ≈ B is presented. As only a basis of the right singular subspace associated with the smallest singular values of the data [A; B] is needed, the computational cost can be reduced considerably by using the partial SVD algorithm. This algorithm computes in an efficient way a basis for the left and/or right singular subspace of a matrix associated with its smallest singular values.An analysis of the operation counts, as well as computational results, show the relative efficiency of PTLS with respect to the classical TLS algorithm. Typically, PTLS reduces the computation time with a factor 2.     

Small sample statistical condition estimation for the total least squares problem

In this paper, under the genericity condition, we study the condition estimation of the total least squares (TLS) problem based on small sample condition estimation (SCE), which can be incorporated into the direct solver for the TLS problem via the singular value decomposition (SVD) of the augmented matrix [A, b]. Our proposed condition estimation algorithms are efficient for the small and medium size TLS problem because they utilize the computed SVD of [A, b] during the numerical solution to the TLS problem. Numerical examples illustrate the reliability of the algorithms. Both normwise and componentwise perturbations are considered. Moreover, structured condition estimations are investigated for the structured TLS problem.     

Computing the CS and the generalized singular value decompositions

Summary If the columns of a matrix are orthonormal and it is partitioned into a 2-by-1 block matrix, then the singular value decompositions of the blocks are related. This is the essence of the CS decomposition. The computation of these related SVD's requires some care. Stewart has given an algorithm that uses the LINPACK SVD algorithm together with a Jacobitype clean-up operation on a cross-product matrix. Our technique is equally stable and fast but avoids the cross product matrix. The simplicity of our technique makes it more amenable to parallel computation on systolic-type computer architectures. These developments are of interest because a good way to compute the generalized singular value decomposition of a matrix pair (A, B) is to compute the CS decomposition of a certain orthogonal column matrix related toA andB.The research associated with this paper was partially supported by the Office of Naval Research contract N00014-83-K-0640, USA     

The generalized singular value decomposition and the general (A-λB)-problem

In the study of the canonical structure of matrix pencilsA-B, the column and row nullities ofA andB and the possible common nullspaces give information about the Kronecker structure ofA-B. It is shown how to extract the significant information concerning these nullspaces from the generalized singular value decomposition (GSVD) of the matrix pair (A, B), These properties are the basis for the RGSVD-algorithm that will be published elsewhere [9]. An algorithm for the numerical computation of the transformation matrices (V, X), used in an equivalence transformationV ^H (A-B)X, is presented and discussed. A generalizedQZ-decomposition (GQZD) of a matrix pair (A, B) is also formulated, giving a unitary equivalent transformationV ^H (A-B)Q. If during the computationsX gets too illconditioned we switch to the GQZD, sacrificing a simple diagonal structure of the transformedB-part in order to maintain stability.Dedicated to Germund Dahlquist, on the occasion of his 60th birthday.     

Low Rank Approximation of a Hankel Matrix by Structured Total Least Norm

The structure preserving rank reduction problem arises in many important applications. The singular value decomposition (SVD), while giving the closest low rank approximation to a given matrix in matrix L ₂ norm and Frobenius norm, may not be appropriate for these applications since it does not preserve the given structure. We present a new method for structure preserving low rank approximation of a matrix, which is based on Structured Total Least Norm (STLN). The STLN is an efficient method for obtaining an approximate solution to an overdetermined linear system AX B, preserving the given linear structure in the perturbation [E F] such that (A + E)X = B + F. The approximate solution can be obtained to minimize the perturbation [E F] in the L _p norm, where p = 1, 2, or . An algorithm is described for Hankel structure preserving low rank approximation using STLN with L _p norm. Computational results are presented, which show performances of the STLN based method for L ₁ and L ₂ norms for reduced rank approximation for Hankel matrices.     

The Role of the Inner Product in Stopping Criteria for Conjugate Gradient Iterations

Two natural and efficient stopping criteria are derived for conjugate gradient (CG) methods, based on iteration parameters. The derivation makes use of the inner product matrix B-defining the CG method. In particular, the relationship between the eigenvalues and B-norm of a matrix is investigated, and it is shown that the ratio of largest to smallest eigenvalues defines the B-condition number of the matrix. Upper and lower bounds on various measures of the error are also given. The compound stopping criterion presented here is an obvious default in software packages because it does not require any additional norm computations.     

Algebraic connections between the least squares and total least squares problems

Summary In this paper the closeness of the total least squares (TLS) and the classical least squares (LS) problem is studied algebraically. Interesting algebraic connections between their solutions, their residuals, their corrections applied to data fitting and their approximate subspaces are proven.All these relationships point out the parameters which mainly determine the equivalences and differences between the two techniques. These parameters also lead to a better understanding of the differences in sensitivity between both approaches with respect to perturbations of the data.In particular, it is shown how the differences between both approaches increase when the equationsAXB become less compatible, when the length ofB orX is growing or whenA tends to be rank-deficient. They are maximal whenB is parallel with the singular vector ofA associated with its smallest singular value. Furthermore, it is shown how TLS leads to a weighted LS problem, and assumptions about the underlying perturbation model of both techniques are deduced. It is shown that many perturbation models correspond with the same TLS solution.Senior Research Assistant of the Belgian N.F.W.O. (National Fund of Scientific Research)     

Robust Hadamard Matrices,Unistochastic Rays in Birkhoff Polytope and Equi-Entangled Bases in Composite Spaces

We study a special class of (real or complex) robust Hadamard matrices, distinguished by the property that their projection onto a 2-dimensional subspace forms a Hadamard matrix. It is shown that such a matrix of order n exists, if there exists a skew Hadamard matrix or a symmetric conference matrix of this size. This is the case for any even \(n\le 20\), and for these dimensions we demonstrate that a bistochastic matrix B located at any ray of the Birkhoff polytope, (which joins the center of this body with any permutation matrix), is unistochastic. An explicit form of the corresponding unitary matrix U, such that \(B_{ij}=|U_{ij}|^2\), is determined by a robust Hadamard matrix. These unitary matrices allow us to construct a family of orthogonal bases in the composed Hilbert space of order \(n \times n\). Each basis consists of vectors with the same degree of entanglement and the constructed family interpolates between the product basis and the maximally entangled basis. In the case \(n=4\) we study geometry of the set \({\mathcal U}_4\) of unistochastic matrices, conjecture that this set is star-shaped and estimate its relative volume in the Birkhoff polytope \({\mathcal B}_4\).     

Weakly Outer Inner Functions

An inner function I in the unit ball B_nB_nⁿ is said to be weakly outer if the closed subspace I H ^p(B ⁿ) is weakly dense in the Hardy space H^p(B ⁿ), 0n for all n1. We also investigate inner functions I such that the subspace IH^p(B ⁿ) is not weakly dense in H^p(B ⁿ).     

A QR Decomposition for Matrix Pencils

This paper describes an efficient and numerically stable modification of the QR decomposition for solving a parametric set of linear least squares problems with a parametric matrix A + B for several values of the parameter . The method is demonstrated on a typical application.     

Norms in quadratic fields and their relations to non commuting 2×2 matrices I

LetA be a fixed 2×2 integral matrix with irrational characteristic roots. LetB be an arbitrary 2×2 integral matrix. It was previously shown that -det (AB-BA)=norm, where is in the field of the characteristic roots ofA. It is now shown that the 's corresponding to varyingB's can be chosen to form a fractional ideal in this field.     

Some inequalities for communtators and an application to spectral variation

Summary If –I is a positive semidefinite operator andA andB are either both Hermitian or both unitary, then every unitarily invariant norm ofA–B is shown to be bounded by that ofA–B. Some related inequalities are proved. An application leads to a generalization of the Lidskii-Wielandt inequality to matrices similar to Hermitian.Dedicated to the memory of Alexander M. Ostrowski on the occasion of the 100th anniversary of his birth     

Inverse subspace problems with applications

Given a square matrix A, the inverse subspace problem is concerned with determining a closest matrix to A with a prescribed invariant subspace. When A is Hermitian, the closest matrix may be required to be Hermitian. We measure distance in the Frobenius norm and discuss applications to Krylov subspace methods for the solution of large‐scale linear systems of equations and eigenvalue problems as well as to the construction of blurring matrices. Extensions that allow the matrix A to be rectangular and applications to Lanczos bidiagonalization, as well as to the recently proposed subspace‐restricted SVD method for the solution of linear discrete ill‐posed problems, also are considered.Copyright © 2013 John Wiley & Sons, Ltd.     

Norms in quadratic fields and their relations to non commuting 2×2 matrices I

LetA be a fixed 2×2 integral matrix with irrational characteristic roots. LetB be an arbitrary 2×2 integral matrix. It was previously shown that -det (AB-BA)=norm, where is in the field of the characteristic roots ofA. It is now shown that the 's corresponding to varyingB's can be chosen to form a fractional ideal in this field.Dedicated to Prof. Dr. E. Hlawka on the occasion of his 60th birthdayThis work was carried out (in part) under an NSF grant.     

Robust tensor completion using transformed tensor singular value decomposition

In this article, we study robust tensor completion by using transformed tensor singular value decomposition (SVD), which employs unitary transform matrices instead of discrete Fourier transform matrix that is used in the traditional tensor SVD. The main motivation is that a lower tubal rank tensor can be obtained by using other unitary transform matrices than that by using discrete Fourier transform matrix. This would be more effective for robust tensor completion. Experimental results for hyperspectral, video and face datasets have shown that the recovery performance for the robust tensor completion problem by using transformed tensor SVD is better in peak signal‐to‐noise ratio than that by using Fourier transform and other robust tensor completion methods.     

Computing Eigenelements of Real Symmetric Matrices via Optimization

In certain circumstances, it is advantageous to use an optimization approach in order to solve the generalized eigenproblem, Ax = Bx, where A and B are real symmetric matrices and B is positive definite. In particular, this is the case when the matrices A and B are very large the computational cost, prohibitive, of solving, with high accuracy, systems of equations involving these matrices. Usually, the optimization approach involves optimizing the Rayleigh quotient.We first propose alternative objective functions to solve the (generalized) eigenproblem via (unconstrained) optimization, and we describe the variational properties of these functions.We then introduce some optimization algorithms (based on one of these formulations) designed to compute the largest eigenpair. According to preliminary numerical experiments, this work could lead the way to practical methods for computing the largest eigenpair of a (very) large symmetric matrix (pair).     

Computing optimal scalings by parametric network algorithms

Asymmetric scaling of a square matrixA 0 is a matrix of the formXAX ^–1 whereX is a nonnegative, nonsingular, diagonal matrix having the same dimension ofA. Anasymmetric scaling of a rectangular matrixB 0 is a matrix of the formXBY ^–1 whereX andY are nonnegative, nonsingular, diagonal matrices having appropriate dimensions. We consider two objectives in selecting a symmetric scaling of a given matrix. The first is to select a scalingA of a given matrixA such that the maximal absolute value of the elements ofA is lesser or equal that of any other corresponding scaling ofA. The second is to select a scalingB of a given matrixB such that the maximal absolute value of ratios of nonzero elements ofB is lesser or equal that of any other corresponding scaling ofB. We also consider the problem of finding an optimal asymmetric scaling under the maximal ratio criterion (the maximal element criterion is, of course, trivial in this case). We show that these problems can be converted to parametric network problems which can be solved by corresponding algorithms.This research was supported by NSF Grant ECS-83-10213.