Perturbation and error analyses for block downdating of a Cholesky decomposition

A new perturbation result is presented for the problem of block downdating a Cholesky decompositionX ^T X = R ^T R. Then, a condition number for block downdating is proposed and compared to other downdating condition numbers presented in literature recently. This new condition number is shown to give a tighter bound in many cases. Using the perturbation theory, an error analysis is presented for the block downdating algorithms based on the LINPACK downdating algorithm and stabilized hyperbolic transformations. An error analysis is also given for block downdating using Corrected Seminormal Equations (CSNE), and it is shown that for ill-conditioned downdates this method gives more accurate results than the algorithms based on the LINPACK downdating algorithm or hyperbolic transformations. We classify the problems for which the CSNE downdating method produces a downdated upper triangular matrix which is comparable in accuracy to the upper triangular factor obtained from the QR decomposition by Householder transformations on the data matrix with the row block deleted.Dedicated to Ji-guang Sun in honour of his 60th birthdayThe work of the second author was supported in part by the National Science Foundation grant CCR-9209726.     

Modified Gram-Schmidt-based methods for block downdating the Cholesky factorization

The hyperbolic modified Gram-Schmidt (HMGS) method is proposed for block downdating the Cholesky factorization. The method might be unsatisfactory due to rounding errors. A modified version based on the MGS process is presented and is shown to be mixed stable. Numerical tests show that the new method has the same numerical properties as the generalized LINPACK-type algorithm, and can work better than the Householder-based algorithm given by Bojanczyk and Steinhardt (1991) [9].     

Fast Algorithm for Cross-Validation of the Best Linear Unbiased Predictor

This article proposes a new algorithm for cross-validation of the best linear unbiased predictor. The algorithm relies on a new technique for downdating the inverse of a Cholesky factor. Given n data points, the new algorithm has complexity O(n³), compared to O(n⁴), which is the order for the more traditional delete one and recalculate method.     

An algorithm and a stability theory for downdating the ULV decomposition

An alternative to performing the singular value decomposition is to factor a matrixA into , whereU andV are orthogonal matrices andC is a lower triangular matrix which indicates a separation between two subspaces by the size of its columns. These subspaces are denoted byV = (V ₁,V ₂), where the columns ofC are partitioned conformally intoC = (C ₁,C ₂) with C _{2 F} . Here is some tolerance. In recent years, this has been called the ULV decomposition (ULVD).If the matrixA results from statistical observations, it is often desired to remove old observations, thus deleting a row fromA and its ULVD. In matrix terms, this is called a downdate. A downdating algorithm is proposed that preserves the structure in the downdated matrix to the extent possible. Strong stability results are proven for these algorithms based upon a new perturbation theory.The research of Jesse L. Barlow and Hongyuan Zha was supported by the National Science Foundation under grant no. CCR-9201612(Barlow) and CCR-9308399(Zha). The research of Peter A. Yoon was supported by the Office of Naval Research under the Fundamental Research Initiatives Program. Peter A. Yoon also has an appointment with the Applied Research Laboratory, The Pennsylvania State University, University Park, PA 16802, USA     

Rank-k modification methods for recursive least squares problems

In least squares problems, it is often desired to solve the same problem repeatedly but with several rows of the data either added, deleted, or both. Methods for quickly solving a problem after adding or deleting one row of data at a time are known. In this paper we introduce fundamental rank-k updating and downdating methods and show how extensions of rank-1 downdating methods based on LINPACK, Corrected Semi-Normal Equations (CSNE), and Gram-Schmidt factorizations, as well as new rank-k downdating methods, can all be derived from these fundamental results. We then analyze the cost of each new algorithm and make comparisons tok applications of the corresponding rank-1 algorithms. We provide experimental results comparing the numerical accuracy of the various algorithms, paying particular attention to the downdating methods, due to their potential numerical difficulties for ill-conditioned problems. We then discuss the computation involved for each downdating method, measured in terms of operation counts and BLAS calls. Finally, we provide serial execution timing results for these algorithms, noting preferable points for improvement and optimization. From our experiments we conclude that the Gram-Schmidt methods perform best in terms of numerical accuracy, but may be too costly for serial execution for large problems.Research supported in part by the Joint Services Electronics Program, contract no. F49620-90-C-0039.     

On the modified Gram-Schmidt algorithm for weighted and constrained linear least squares problems

A framework and an algorithm for using modified Gram-Schmidt for constrained and weighted linear least squares problems is presented. It is shown that a direct implementation of a weighted modified Gram-Schmidt algorithm is unstable for heavily weighted problems. It is shown that, in most cases it is possible to get a stable algorithm by a simple modification free from any extra computational costs. In particular, it is not necessary to perform reorthogonalization.Solving the weighted and constrained linear least squares problem with the presented weighted modified Gram-Schmidt algorithm is seen to be numerically equivalent to an algorithm based on a weighted Householder-likeQR factorization applied to a slightly larger problem. This equivalence is used to explain the instability of the weighted modified Gram-Schmidt algorithm. If orthogonality, with respect to a weighted inner product, of the columns inQ is important then reorthogonalization can be used. One way of performing such reorthogonalization is described.Computational tests are given to show the main features of the algorithm.     

Time-stepping and preserving orthonormality

Certain applications produce initial value ODEs whose solutions, regarded as time-dependent matrices, preserve orthonormality. Such systems arise in the computation of Lyapunov exponents and the construction of smooth singular value decompositions of parametrized matrices. For some special problem classes, there exist time-stepping methods that automatically inherit the orthonormality preservation. However, a more widely applicable approach is to apply a standard integrator and regularly replace the approximate solution by an orthonormal matrix. Typically, the approximate solution is replaced by the factorQ from its QR decomposition (computed, for example, by the modified Gram-Schmidt method). However, the optimal replacement—the one that is closest in the Frobenius norm—is given by the orthonormal polar factor. Quadratically convergent iteration schemes can be used to compute this factor. In particular, there is a matrix multiplication based iteration that is ideally suited to modern computer architectures. Hence, we argue that perturbing towards the orthonormal polar factor is an attractive choice, and we consider performing a fixed number of iterations. Using the optimality property we show that the perturbations improve the departure from orthonormality without significantly degrading the finite-time global error bound for the ODE solution. Our analysis allows for adaptive time-stepping, where a local error control process is driven by a user-supplied tolerance. Finally, using a recent result of Sun, we show how the global error bound carries through to the case where the orthonormal QR factor is used instead of the orthonormal polar factor. This work was supported by Engineering and Physical Sciences Research Council grants GR/H94634 and GR/K80228.     

Modifying the generalized singular value decomposition with application with application in direction-of-arrival finding

We consider updating and downdating problems for the generalized singular value decomposition (GSVD) of matrix pairs when new rows are added to one of the matrices or old rows are deleted. Two classes of algorithms are developed, one based on the CS decomposition formulation of the GSVD and the other based on the generalized eigenvalue decomposition formulation. In both cases we show that the updating and downdating problems can be reduced to a rank-one SVD updating problem. We also provide perturbation analysis for the cases when the added or deleted rows are subject to errors. Numerical experiments on direction-of-arrival (DOA) finding with colored noises are carried out to demonstrate the tracking ability of the algorithms. The work of the first author was supported in part by NSF grants CCR-9308399 and CCR-9619452. The work of the second author was supported in part by China State Major Key Project for Basic Researches.     

CGS algorithms for unconstrained minimization of functions

In 1952, Hestenes and Stiefel first established, along with the conjugate-gradient algorithm, fundamental relations which exist between conjugate direction methods for function minimization on the one hand and Gram-Schmidt processes relative to a given positive-definite, symmetric matrix on the other. This paper is based on a recent reformulation of these relations by Hestenes which yield the conjugate Gram-Schmidt (CGS) algorithm. CGS includes a variety of function minimization routines, one of which is the conjugate-gradient routine. This paper gives the basic equations of CGS, including the form applicable to minimizing general nonquadratic functions ofn variables. Results of numerical experiments of one form of CGS on five standard test functions are presented. These results show that this version of CGS is very effective.The preparation of this paper was sponsored in part by the US Army Research Office, Grant No. DH-ARO-D-31-124-71-G18.The authors wish to thank Mr. Paul Speckman for the many computer runs made using these algorithms. They served as a good check on the results which they had obtained earlier. Special thanks must go to Professor M. R. Hestenes whose constant encouragement and assistance made this paper possible.     

Updating and downdating an upper trapezoidal sparse orthogonal factorization{dagger}

^** Email: santos{at}ctima.uma.es^*** Corresponding Author. Email: pablito{at}ctima.uma.es We describe how to update and downdate an upper trapezoidalsparse orthogonal factorization, namely the sparse QR factorizationof A_k^T, where A_k is a ‘tall and thin’ full columnrank matrix formed with a subset of the columns of a fixed matrixA. In order to do this, we have adapted Saunders' techniquesof the early 1970s for square matrices, to rectangular matrices(with fewer columns than rows) by using the static data structureof George and Heath of the early 1980s but allowing row downdatingon it. An implicitly determined column permutation allows usto dispense with computing a new ordering after each update/downdate;it fits well into the LINPACK downdating algorithm and ensuresthat the updated trapezoidal factor will remain sparse. We giveall the necessary formulae even if the orthogonal factor isnot available, and we comment on our implementation using thesparse toolbox of MATLAB 5.     

Parallel QR decomposition of a rectangular matrix

Summary We show that the greedy algorithm introduced in [1] and [5] to perform the parallel QR decomposition of a dense rectangular matrix of sizem×n is optimal. Then we assume thatm/n ² tends to zero asm andn go to infinity, and prove that the complexity of such a decomposition is asymptotically2n, when an unlimited number of processors is available.     

A numerical procedure for computing the Moore-Penrose inverse

An algorithm for computing the Moore-Penrose inverse of an arbitraryn×m real matrixA is presented which uses a Gram-Schmidt like procedure to form anA-orthogonal set of vectors which span the subspace perpendicular to the kernel ofA. This one procedure will work for any value ofn andm, and for any value of rank (A).     

A note on minimal matrix representation of closure operations

A matrixM withn columns represents a closure operationF(A), (A⊂X, |X|=n) if for anyA, any two rows equal in the columns corresponding toA are also equal inF(A). Letm(F) be the minimum number of rows of the matrices representingF. Lower and upper estimates on maxm(F) are given where max runs over the set of all closure operations onn elements.     

A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization

 In this paper, we present a nonlinear programming algorithm for solving semidefinite programs (SDPs) in standard form. The algorithm's distinguishing feature is a change of variables that replaces the symmetric, positive semidefinite variable X of the SDP with a rectangular variable R according to the factorization X=RR ^T . The rank of the factorization, i.e., the number of columns of R, is chosen minimally so as to enhance computational speed while maintaining equivalence with the SDP. Fundamental results concerning the convergence of the algorithm are derived, and encouraging computational results on some large-scale test problems are also presented. Received: March 22, 2001 / Accepted: August 30, 2002 Published online: December 9, 2002 Key Words. semidefinite programming – low-rank factorization – nonlinear programming – augmented Lagrangian – limited memory BFGS This research was supported in part by the National Science Foundation under grants CCR-9902010, INT-9910084, CCR-0203426 and CCR-0203113     

A NUMERICALLY STABLE BLOCK MODIFIED GRAM-SCHMIDT ALGORITHM FOR SOLVING STIFF WEIGHTED LEAST SQUARES PROBLEMS

Recently, Wei in proved that perturbed stiff weighted pseudoinverses and stiff weighted least squares problems are stable, if and only if the original and perturbed coefficient matrices A and A^- satisfy several row rank preservation conditions. According to these conditions, in this paper we show that in general, ordinary modified Gram-Schmidt with column pivoting is not numerically stable for solving the stiff weighted least squares problem. We then propose a row block modified Gram-Schmidt algorithm with column pivoting, and show that with appropriately chosen tolerance, this algorithm can correctly determine the numerical ranks of these row partitioned sub-matrices, and the computed QR factor R^- contains small roundoff error which is row stable. Several numerical experiments are also provided to compare the results of the ordinary Modified Gram-Schmidt algorithm with column pivoting and the row block Modified Gram-Schmidt algorithm with column pivoting.     

A modification to the linpack downdating algorithm

Alinpack downdating algorithm is being modified by interleaving its two different phases, the forward solving a triangular system and the backward sweep of Givens rotations, to yield a new forward method for finding the Cholesky decomposition ofR ^T R –zz ^T . By showing that the new algorithm saves forty percent purely redundant operations of the original, better stability properties are expected. In addition, various other downdating algorithms are rederived and analyzed under a uniform framework.     

Implicit QR algorithms for palindromic and even eigenvalue problems

In the spirit of the Hamiltonian QR algorithm and other bidirectional chasing algorithms, a structure-preserving variant of the implicit QR algorithm for palindromic eigenvalue problems is proposed. This new palindromic QR algorithm is strongly backward stable and requires less operations than the standard QZ algorithm, but is restricted to matrix classes where a preliminary reduction to structured Hessenberg form can be performed. By an extension of the implicit Q theorem, the palindromic QR algorithm is shown to be equivalent to a previously developed explicit version. Also, the classical convergence theory for the QR algorithm can be extended to prove local quadratic convergence. We briefly demonstrate how even eigenvalue problems can be addressed by similar techniques. D. S. Watkins partly supported by Deutsche Forschungsgemeinschaft through Matheon, the DFG Research Center Mathematics for key technologies in Berlin.     

最佳逼近的正交化算法

In the present paper,we shall give a new algorithm of the best approximation in Hilbert spaces by using Gram-Schmidt orthogonalization and give some examples to show that the new method is simple and convenient.And we also point out that the best approximation have a wonderful superposition property by using orthogonal method.     

Improved Gram–Schmidt Type Downdating Methods

The problem of deleting a row from a Q–R factorization (called downdating) using Gram–Schmidt orthogonalization is intimately connected to using classical iterative methods to solve a least squares problem with the orthogonal factor as the coefficient matrix. Past approaches to downdating have focused upon accurate computation of the residual of that least squares problem, then finding a unit vector in the direction of the residual that becomes a new column for the orthogonal factor. It is also important to compute the solution vector of the related least squares problem accurately, as that vector must be used in the downdating process to maintain good backward error in the new factorization. Using this observation, new algorithms are proposed. One of the new algorithms proposed is a modification of one due to Yoo and Park [BIT, 36:161–181, 1996]. That algorithm is shown to be a Gram–Schmidt procedure. Also presented are new results that bound the loss of orthogonality after downdating. An error analysis shows that the proposed algorithms’ behavior in floating point arithmetic is close to their behavior in exact arithmetic. Experiments show that the changes proposed in this paper can have a dramatic impact upon the accuracy of the downdated Q–R decomposition. AMS subject classification (2000) 65F20, 65F25     

Modifiable low‐rank approximation to a matrix

A truncated ULV decomposition (TULVD) of an m×n matrix X of rank k is a decomposition of the form X = ULV^T+E, where U and V are left orthogonal matrices, L is a k×k non‐singular lower triangular matrix, and E is an error matrix. Only U,V, L, and ∥E∥_F are stored, but E is not stored. We propose algorithms for updating and downdating the TULVD. To construct these modification algorithms, we also use a refinement algorithm based upon that in (SIAM J. Matrix Anal. Appl. 2005; 27 (1):198–211) that reduces ∥E∥_F, detects rank degeneracy, corrects it, and sharpens the approximation. Copyright © 2009 John Wiley & Sons, Ltd.