Fast Iterative Solution of Saddle Point Problems in Optimal Control Based on Wavelets

In this paper, wavelet techniques are employed for the fast numerical solution of a control problem governed by an elliptic boundary value problem with boundary control. A quadratic cost functional involving natural norms of the state and the control is to be minimized. Firstly the constraint, the elliptic boundary value problem, is formulated in an appropriate weak form that allows to handle varying boundary conditions explicitly: the boundary conditions are treated by Lagrange multipliers, leading to a saddle point problem. This is combined with a fictitious domain approach in order to cover also more complicated boundaries.Deviating from standard approaches, we then use (biorthogonal) wavelets to derive an equivalent infinite discretized control problem which involves only ₂-norms and -operators. Classical methods from optimization yield the corresponding optimality conditions in terms of two weakly coupled (still infinite) saddle point problems for which a unique solution exists. For deriving finite-dimensional systems which are uniformly invertible, stability of the discretizations has to be ensured. This together with the ₂-setting circumvents the problem of preconditioning: all operators have uniformly bounded condition numbers independent of the discretization.In order to numerically solve the resulting (finite-dimensional) linear system of the weakly coupled saddle point problems, a fully iterative method is proposed which can be viewed as an inexact gradient scheme. It consists of a gradient algorithm as an outer iteration which alternatingly picks the two saddle point problems, and an inner iteration to solve each of the saddle point problems, exemplified in terms of the Uzawa algorithm. It is proved here that this strategy converges, provided that the inner systems are solved sufficiently well. Moreover, since the system matrix is well-conditioned, it is shown that in combination with a nested iteration strategy this iteration is asymptotically optimal in the sense that it provides the solution on discretization level J with an overall amount of arithmetic operations that is proportional to the number of unknows N _J on that level.Finally, numerical results are provided.     

A comparison of abstract versions of deflation,balancing and additive coarse grid correction preconditioners

In this paper we consider various preconditioners for the conjugate gradient (CG) method to solve large linear systems of equations with symmetric positive definite system matrix. We continue the comparison between abstract versions of the deflation, balancing and additive coarse grid correction preconditioning techniques started in (SIAM J. Numer. Anal. 2004; 42 :1631–1647; SIAM J. Sci. Comput. 2006; 27 :1742–1759). There the deflation method is compared with the abstract additive coarse grid correction preconditioner and the abstract balancing preconditioner. Here, we close the triangle between these three methods. First of all, we show that a theoretical comparison of the condition numbers of the abstract additive coarse grid correction and the condition number of the system preconditioned by the abstract balancing preconditioner is not possible. We present a counter example, for which the condition number of the abstract additive coarse grid correction preconditioned system is below the condition number of the system preconditioned with the abstract balancing preconditioner. However, if the CG method is preconditioned by the abstract balancing preconditioner and is started with a special starting vector, the asymptotic convergence behavior of the CG method can be described by the so‐called effective condition number with respect to the starting vector. We prove that this effective condition number of the system preconditioned by the abstract balancing preconditioner is less than or equal to the condition number of the system preconditioned by the abstract additive coarse grid correction method. We also provide a short proof of the relationship between the effective condition number and the convergence of CG. Moreover, we compare the A‐norm of the errors of the iterates given by the different preconditioners and establish the orthogonal invariants of all three types of preconditioners. Copyright © 2008 John Wiley & Sons, Ltd.     

Analysis and implementation of a dual algorithm for constrained optimization

This paper analyzes a constrained optimization algorithm that combines an unconstrained minimization scheme like the conjugate gradient method, an augmented Lagrangian, and multiplier updates to obtain global quadratic convergence. Some of the issues that we focus on are the treatment of rigid constraints that must be satisfied during the iterations and techniques for balancing the error associated with constraint violation with the error associated with optimality. A preconditioner is constructed with the property that the rigid constraints are satisfied while ill-conditioning due to penalty terms is alleviated. Various numerical linear algebra techniques required for the efficient implementation of the algorithm are presented, and convergence behavior is illustrated in a series of numerical experiments.This research was supported by the National Science Foundation Grant DMS-89-03226 and by the U.S. Army Research Office Contract DAA03-89-M-0314.We thank the referees for their many perceptive comments which led to substantial improvements in the presentation of this paper.     

A method for finite element parallel viscous compressible flow calculations

This paper presents the parallelization aspects of a solution method for the fully coupled 3D compressible Navier-Stokes equations. The algorithmic thrust of the approach, embedded in a finite element code NS3D, is the linearization of the governing equations through Newton methods, followed by a fully coupled solution of velocities and pressure at each non-linear iteration by preconditioned conjugate gradient-like iterative algorithms. For the matrix assembly, as well as for the linear equation solver, efficient coarse-grain parallel schemes have been developed for shared memory machines, as well as for networks of workstations, with a moderate number of processors. The parallel iterative schemes, in particular, circumvent some of the difficulties associated with domain decomposition methods, such as geometry bookkeeping and the sometimes drastic convergence slow-down of partitioned non-linear problems.     

An overlapping domain decomposition preconditioner for a class of discontinuous Galerkin approximations of advection-diffusion problems

We consider a scalar advection-diffusion problem and a recently proposed discontinuous Galerkin approximation, which employs discontinuous finite element spaces and suitable bilinear forms containing interface terms that ensure consistency. For the corresponding sparse, nonsymmetric linear system, we propose and study an additive, two-level overlapping Schwarz preconditioner, consisting of a coarse problem on a coarse triangulation and local solvers associated to a family of subdomains. This is a generalization of the corresponding overlapping method for approximations on continuous finite element spaces. Related to the lack of continuity of our approximation spaces, some interesting new features arise in our generalization, which have no analog in the conforming case. We prove an upper bound for the number of iterations obtained by using this preconditioner with GMRES, which is independent of the number of degrees of freedom of the original problem and the number of subdomains. The performance of the method is illustrated by several numerical experiments for different test problems using linear finite elements in two dimensions.

     

Preconditioners Based on Fundamental Solutions

We consider a new preconditioning technique for the iterative solution of linear systems of equations that arise when discretizing partial differential equations. The method is applied to finite difference discretizations, but the ideas apply to other discretizations too. If E is a fundamental solution of a differential operator P, we have E*(Pu) = u. Inspired by this, we choose the preconditioner to be a discretization of an approximate inverse K, given by a convolution-like operator with E as a kernel. We present analysis showing that if P is a first order differential operator, KP is bounded, and numerical results show grid independent convergence for first order partial differential equations, using fixed point iterations. For the second order convection-diffusion equation convergence is no longer grid independent when using fixed point iterations, a result that is consistent with our theory. However, if the grid is chosen to give a fixed number of grid points within boundary layers, the number of iterations is independent of the physical viscosity parameter. AMS subject classification (2000) 65F10, 65N22     

A Class of Incomplete Orthogonal Factorization Methods. II: Implementation and Results

We present, implement and test several incomplete QR factorization methods based on Givens rotations for sparse square and rectangular matrices. For square systems, the approximate QR factors are used as right-preconditioners for GMRES, and their performance is compared to standard ILU techniques. For rectangular matrices corresponding to linear least-squares problems, the approximate R factor is used as a right-preconditioner for CGLS. A comprehensive discussion is given about the uses, advantages and shortcomings of the preconditioners. AMS subject classification (2000) 65F10, 65F25, 65F50.Received May 2002. Revised October 2004. Communicated by Åke Björck.     

On the preconditioning of matrices with skew-symmetric splittings

The rates of convergence of iterative methods with standard preconditioning techniques usually degrade when the skew-symmetric part S of the matrix is relatively large. In this paper, we address the issue of preconditioning matrices with such large skew-symmetric parts. The main idea of the preconditioner is to split the matrix into its symmetric and skew-symmetric parts and to invert the (shifted) skew-symmetric matrix. Successful use of the method requires the solution of a linear system with matrix I+S. An efficient method is developed using the normal equations, preconditioned by an incomplete orthogonal factorization.Numerical experiments on various systems arising in physics show that the reduction in terms of iteration count compensates for the additional work per iteration when compared to standard preconditioners.     

Dual techniques for constrained optimization

Algorithms to solve constrained optimization problems are derived. These schemes combine an unconstrained minimization scheme like the conjugate gradient method, an augmented Lagrangian, and multiplier updates to obtain global quadratic convergence. Since an augmented Lagrangian can be ill conditioned, a preconditioning strategy is developed to eliminate the instabilities associated with the penalty term. A criterion for deciding when to increase the penalty is presented.This work was supported by the National Science Foundation, Grant Nos. MCS-81-01892, DMS-84-01758, and DMS-85-20926, and by the Air Force Office of Scientific Research, Grant No. AFOSR-ISSA-860091.     

Efficient preconditioning of linear systems arising from the discretization of hyperbolic conservation laws

In this paper, we describe a novel formulation of a preconditioned BiCGSTAB algorithm for the solution of ill-conditioned linear systems Ax=b. The developed extension enables the control of the residual r _m =b–Ax _m of the approximate solution x _m independent of the specific left, right or two-sided preconditioning technique considered. Thereby, the presented modification does not require any additional computational effort and can be introduced directly into existing computer codes. Furthermore, the proceeding is not restricted to the BiCGSTAB method, hence the strategy can serve as a guideline to extend similar Krylov sub-space methods in the same manner. Based on the presented algorithm, we study the behavior of different preconditioning techniques. We introduce a new physically motivated approach within an implicit finite volume scheme for the system of the Euler equations of gas dynamics which is a typical representative of hyperbolic conservation laws. Thereupon a great variety of realistic flow problems are considered in order to give reliable statements concerning the efficiency and performance of modern preconditioning techniques.