Hierarchical Schur complement preconditioner for the stochastic Galerkin finite element methods

Use of the stochastic Galerkin finite element methods leads to large systems of linear equations obtained by the discretization of tensor product solution spaces along their spatial and stochastic dimensions. These systems are typically solved iteratively by a Krylov subspace method. We propose a preconditioner, which takes an advantage of the recursive hierarchy in the structure of the global matrices. In particular, the matrices posses a recursive hierarchical two‐by‐two structure, with one of the submatrices block diagonal. Each of the diagonal blocks in this submatrix is closely related to the deterministic mean‐value problem, and the action of its inverse is in the implementation approximated by inner loops of Krylov iterations. Thus, our hierarchical Schur complement preconditioner combines, on each level in the approximation of the hierarchical structure of the global matrix, the idea of Schur complement with loops for a number of mutually independent inner Krylov iterations, and several matrix–vector multiplications for the off‐diagonal blocks. Neither the global matrix nor the matrix of the preconditioner need to be formed explicitly. The ingredients include only the number of stiffness matrices from the truncated Karhunen–Loève expansion and a good preconditioned for the mean‐value deterministic problem. We provide a condition number bound for a model elliptic problem, and the performance of the method is illustrated by numerical experiments. Copyright © 2013 John Wiley & Sons, Ltd.     

Block Two-stage Methods for Singular Systems and Markov Chains

The use of block two-stage methods for the iterative solution of consistent singular linear systems is studied. In these methods, suitable for parallel computations, different blocks, i.e., smaller linear systems, can be solved concurrently by different processors. Each of these smaller systems are solved by an (inner) iterative method. Hypotheses are provided for the convergence of non-stationary methods, i.e., when the number of inner iterations may vary from block to block and from one outer iteration to another. It is shown that the iteration matrix corresponding to one step of the block method is convergent, i.e., that its powers converge to a limit matrix. A theorem on the convergence of the infinite product of matrices with the same eigenspace corresponding to the eigenvalue 1 is proved, and later used as a tool in the convergence analysis of the block method. The methods studied can be used to solve any consistent singular system, including discretizations of certain differential equations. They can also be used to find stationary probability distribution of Markov chains. This last application is considered in detail.     

Computational Krylov‐based methods for large‐scale differential Sylvester matrix problems

In the present paper, we propose Krylov‐based methods for solving large‐scale differential Sylvester matrix equations having a low‐rank constant term. We present two new approaches for solving such differential matrix equations. The first approach is based on the integral expression of the exact solution and a Krylov method for the computation of the exponential of a matrix times a block of vectors. In the second approach, we first project the initial problem onto a block (or extended block) Krylov subspace and get a low‐dimensional differential Sylvester matrix equation. The latter problem is then solved by some integration numerical methods such as the backward differentiation formula or Rosenbrock method, and the obtained solution is used to build the low‐rank approximate solution of the original problem. We give some new theoretical results such as a simple expression of the residual norm and upper bounds for the norm of the error. Some numerical experiments are given in order to compare the two approaches.     

Least squares solution with the minimum-norm to general matrix equations via iteration

Two iterative algorithms are presented in this paper to solve the minimal norm least squares solution to a general linear matrix equations including the well-known Sylvester matrix equation and Lyapunov matrix equation as special cases. The first algorithm is based on the gradient based searching principle and the other one can be viewed as its dual form. Necessary and sufficient conditions for the step sizes in these two algorithms are proposed to guarantee the convergence of the algorithms for arbitrary initial conditions. Sufficient condition that is easy to compute is also given. Moreover, two methods are proposed to choose the optimal step sizes such that the convergence speeds of the algorithms are maximized. Between these two methods, the first one is to minimize the spectral radius of the iteration matrix and explicit expression for the optimal step size is obtained. The second method is to minimize the square sum of the F-norm of the error matrices produced by the algorithm and it is shown that the optimal step size exits uniquely and lies in an interval. Several numerical examples are given to illustrate the efficiency of the proposed approach.     

块三对角阵分解因子的估值与应用

1.引 言 许多物理应用问题归结为求微分方程数值解,而这可以通过离散化为求解稀疏线性方程组,所以稀疏线性方程组求解的有效性在很大程度上决定了原问题求解算法的有效性.直接     

Geometry of Weyl theory for Jacobi matrices with matrix entries

A Jacobi matrix with matrix entries is a self-adjoint block tridiagonal matrix with invertible blocks on the off-diagonals. The Weyl surface describing the dependence of Green’s matrix on the boundary conditions is interpreted as the set of maximally isotropic subspaces of a quadratic form given by the Wronskian. Analysis of the possibly degenerate limit quadratic form leads to the limit point/limit surface theory of maximal symmetric extensions for semi-infinite Jacobi matrices with matrix entries with arbitrary deficiency indices. The resolvent of the extensions is calculated explicitly.     

A wavelet operational method for solving fractional partial differential equations numerically

Fractional calculus is an extension of derivatives and integrals to non-integer orders, and a partial differential equation involving the fractional calculus operators is called the fractional PDE. They have many applications in science and engineering. However not only the analytical solution existed for a limited number of cases, but also the numerical methods are very complicated and difficult. In this paper, we newly establish the simulation method based on the operational matrices of the orthogonal functions. We formulate the operational matrix of integration in a unified framework. By using the operational matrix of integration, we propose a new numerical method for linear fractional partial differential equation solving. In the method, we (1) use the Haar wavelet; (2) establish a Lyapunov-type matrix equation; and (3) obtain the algebraic equations suitable for computer programming. Two examples are given to demonstrate the simplicity, clarity and powerfulness of the new method.     

Invertibility of matrices with almost-commuting entries

In this article, formal determinants of matrices over noncommnutative rings are examined for information about matrix invertibility. In this situation determinants are of course no longer multiplicative functions. The model to be generalized is the classical theorem of Schar for matrices in block form whose blocks below the first row commute. It will be seen that Schur's theorem holds over a very wide class of rings for matrices of arbitrary size. Additionally, the results provide a test for a matrix to be a non zero divisor; and by symmetry a permanental condition for invertibility of certain matrices is obtained.     

Invertibility of matrices with almost-commuting entries

In this article, formal determinants of matrices over noncommnutative rings are examined for information about matrix invertibility. In this situation determinants are of course no longer multiplicative functions. The model to be generalized is the classical theorem of Schar for matrices in block form whose blocks below the first row commute. It will be seen that Schur's theorem holds over a very wide class of rings for matrices of arbitrary size. Additionally, the results provide a test for a matrix to be a non zero divisor; and by symmetry a permanental condition for invertibility of certain matrices is obtained.     

Generating equations approach for quadratic matrix equations

We show how Van Loan's method for annulling the (2,1) block of skew‐Hamiltonian matrices by symplectic‐orthogonal similarity transformation generalizes to general matrices and provides a numerical algorithm for solving the general quadratic matrix equation: For skew‐Hamiltonian matrices we find their canonical form under a similarity transformation and find the class of all symplectic‐orthogonal similarity transformations for annulling the (2,1) block and simultaneously bringing the (1,1) block to Hessenberg form. We present a structure‐preserving algorithm for the solution of continuous‐time algebraic Riccati equation. Unlike other methods in the literature, the final transformed Hamiltonian matrix is not in Hamiltonian–Schur form. Three applications are presented: (a) for a special system of partial differential equations of second order for a single unknown function, we obtain the matrix of partial derivatives of second order of the unknown function by only algebraic operations and differentiation of functions; (b) for a similar transformation of a complex matrix into a symmetric (and three‐diagonal) one by applying only finite algebraic transformations; and (c) for finite‐step reduction of the eigenvalues–eigenvectors problem of a Hermitian matrix to the eigenvalues– eigenvectors problem of a real symmetric matrix of the same dimension. Copyright © 1999 John Wiley & Sons, Ltd.     

Finite iterative solutions to coupled Sylvester-conjugate matrix equations

This paper is concerned with solutions to the so-called coupled Sylveter-conjugate matrix equations, which include the generalized Sylvester matrix equation and coupled Lyapunov matrix equation as special cases. An iterative algorithm is constructed to solve this kind of matrix equations. By using the proposed algorithm, the existence of a solution to a coupled Sylvester-conjugate matrix equation can be determined automatically. When the considered matrix equation is consistent, it is proven by using a real inner product in complex matrix spaces as a tool that a solution can be obtained within finite iteration steps for any initial values in the absence of round-off errors. Another feature of the proposed algorithm is that it can be implemented by using original coefficient matrices, and does not require to transform the coefficient matrices into any canonical forms. The algorithm is also generalized to solve a more general case. Two numerical examples are given to illustrate the effectiveness of the proposed methods.     

Perturbation theory of selfadjoint matrices and sign characteristics under generic structured rank one perturbations

For selfadjoint matrices in an indefinite inner product, possible canonical forms are identified that arise when the matrix is subjected to a selfadjoint generic rank one perturbation. Genericity is understood in the sense of algebraic geometry. Special attention is paid to the perturbation behavior of the sign characteristic. Typically, under such a perturbation, for every given eigenvalue, the largest Jordan block of the eigenvalue is destroyed and (in case the eigenvalue is real) all other Jordan blocks keep their sign characteristic. The new eigenvalues, i.e. those eigenvalues of the perturbed matrix that are not eigenvalues of the original matrix, are typically simple, and in some cases information is provided about their sign characteristic (if the new eigenvalue is real). The main results are proved by using the well known canonical forms of selfadjoint matrices in an indefinite inner product, a version of the Brunovsky canonical form and on general results concerning rank one perturbations obtained.     

Elimination with Noninvertible Pivots

A method for carrying out the Gauss elimination solution of linear systems is presented. The novelty arises from the fact that the pivot matrices are not required to be invertible, so that, for example, a scalar pivot may be zero, and a matrix pivot may be rectangular or singular. The need to execute the elimination algorithm in such circumstances arose in connection with a finite element solution of the Navier-Stokes equations in velocity-pressure variables. This application is briefly discussed, as is a method for the implementation of the algorithm.     

On the convergence of the block principal pivotal algorithm for the LCP

We show that the block principal pivot algorithm (BPPA) for the linear complementarity problem (LCP) solves the problem for a special class of matrices in at most n block principal pivot steps. We provide cycling examples for the BPPA in which the matrix is positive definite or symmetric positive definite. For LCP of order three, we prove that strict column (row) diagonal dominance is a sufficient condition to avoid cycling.     

Minimizing the condition number of a positive definite matrix by completion

Summary. We consider the problem of minimizing the spectral condition number of a positive definite matrix by completion: \noindent where is an Hermitian positive definite matrix, a matrix and is a free Hermitian matrix. We reduce this problem to an optimization problem for a convex function in one variable. Using the minimal solution of this problem we characterize the complete set of matrices that give the minimum condition number. Received October 15, 1993     

Block rearranging elements within matrix columns to minimize the variability of the row sums

Several problems in operations research, such as the assembly line crew scheduling problem and the k-partitioning problem can be cast as the problem of finding the intra-column rearrangement (permutation) of a matrix such that the row sums show minimum variability. A necessary condition for optimality of the rearranged matrix is that for every block containing one or more columns it must hold that its row sums are oppositely ordered to the row sums of the remaining columns. We propose the block rearrangement algorithm with variance equalization (BRAVE) as a suitable method to achieve this situation. It uses a carefully motivated heuristic—based on an idea of variance equalization—to find optimal blocks of columns and rearranges them. When applied to the number partitioning problem, we show that BRAVE outperforms the well-known greedy algorithm and the Karmarkar–Karp differencing algorithm.     

Rank decomposition in zero pattern matrix algebras

For a block upper triangular matrix, a necessary and sufficient condition has been given to let it be the sum of block upper rectangular matrices satisfying certain rank constraints; see H.Bart, A.P.M.Wagelmans (2000). The proof involves elements from integer programming and employs Farkas’ lemma. The algebra of block upper triangular matrices can be viewed as a matrix algebra determined by a pattern of zeros. The present note is concerned with the question whether the decomposition result referred to above can be extended to other zero pattern matrix algebras. It is shown that such a generalization does indeed hold for certain digraphs determining the pattern of zeros. The digraphs in question can be characterized in terms of forests, i.e., disjoint unions of rooted trees.     

On computing of block ILU preconditioner for block tridiagonal systems

We present the recurrence formulas for computing the approximate inverse factors of tridiagonal and pentadiagonal matrices using bordering technique. Resulting algorithms are used to approximate the inverse of pivot blocks needed for constructing block ILU preconditioners for solving the block tridiagonal linear systems, arising from discretization of partial differential equations. Resulting preconditioners are suitable for parallel implementation. Comparison with other methods are also included.     

A time-domain decomposition iterative method for the solution of distributed linear quadratic optimal control problems

We study a class of time-domain decomposition-based methods for the numerical solution of large-scale linear quadratic optimal control problems. Our methods are based on a multiple shooting reformulation of the linear quadratic optimal control problem as a discrete-time optimal control (DTOC) problem. The optimality conditions for this DTOC problem lead to a linear block tridiagonal system. The diagonal blocks are invertible and are related to the original linear quadratic optimal control problem restricted to smaller time-subintervals. This motivates the application of block Gauss–Seidel (GS)-type methods for the solution of the block tridiagonal systems. Numerical experiments show that the spectral radii of the block GS iteration matrices are larger than one for typical applications, but that the eigenvalues of the iteration matrices decay to zero fast. Hence, while the GS method is not expected to convergence for typical applications, it can be effective as a preconditioner for Krylov-subspace methods. This is confirmed by our numerical tests.A byproduct of this research is the insight that certain instantaneous control techniques can be viewed as the application of one step of the forward block GS method applied to the DTOC optimality system.     

A factorization with update procedures for a KKT matrix arising in direct optimal control

Quadratic programs obtained for optimal control problems of dynamic or discrete-time processes usually involve highly block structured Hessian and constraints matrices, to be exploited by efficient numerical methods. In interior point methods, this is elegantly achieved by the widespread availability of advanced sparse symmetric indefinite factorization codes. For active set methods, however, conventional dense matrix techniques suffer from the need to update base matrices in every active set iteration, thereby loosing the sparsity structure after a few updates. This contribution presents a new factorization of a KKT matrix arising in active set methods for optimal control. It fully respects the block structure without any fill-in. For this factorization, matrix updates are derived for all cases of active set changes. This allows for the design of a highly efficient block structured active set method for optimal control and model predictive control problems with long horizons or many control parameters.