Preconditioning Newton–Krylov methods in nonconvex large scale optimization

We consider an iterative preconditioning technique for non-convex large scale optimization. First, we refer to the solution of large scale indefinite linear systems by using a Krylov subspace method, and describe the iterative construction of a preconditioner which does not involve matrices products or matrices storage. The set of directions generated by the Krylov subspace method is used, as by product, to provide an approximate inverse preconditioner. Then, we experience our preconditioner within Truncated Newton schemes for large scale unconstrained optimization, where we generalize the truncation rule by Nash–Sofer (Oper. Res. Lett. 9:219–221, 1990) to the indefinite case, too. We use a Krylov subspace method to both approximately solve the Newton equation and to construct the preconditioner to be used at the current outer iteration. An extensive numerical experience shows that the proposed preconditioning strategy, compared with the unpreconditioned strategy and PREQN (Morales and Nocedal in SIAM J. Optim. 10:1079–1096, 2000), may lead to a reduction of the overall inner iterations. Finally, we show that our proposal has some similarities with the Limited Memory Preconditioners (Gratton et al. in SIAM J. Optim. 21:912–935, 2011).     

A RELAXED HSS PRECONDITIONER FOR SADDLE POINT PROBLEMS FROM MESHFREE DISCRETIZATION＊

In this paper, a relaxed Hermitian and skew-Hermitian splitting （RHSS） preconditioner is proposed for saddle point problems from the element-free Galerkin （EFG） discretization method. The EFG method is one of the most widely used meshfree methods for solving partial differential equations. The RHSS preconditioner is constructed much closer to the coefficient matrix than the well-known HSS preconditioner, resulting in a RHSS fixed-point iteration. Convergence of the RHSS iteration is analyzed and an optimal parameter, which minimizes the spectral radius of the iteration matrix is described. Using the RHSS pre- conditioner to accelerate the convergence of some Krylov subspace methods （like GMRES） is also studied. Theoretical analyses show that the eigenvalues of the RHSS precondi- tioned matrix are real and located in a positive interval. Eigenvector distribution and an upper bound of the degree of the minimal polynomial of the preconditioned matrix are obtained. A practical parameter is suggested in implementing the RHSS preconditioner. Finally, some numerical experiments are illustrated to show the effectiveness of the new preconditioner.     

A multigrid-based preconditioned Krylov subspace method for the Helmholtz equation with PML

In this paper, we generalize the complex shifted Laplacian preconditioner to the complex shifted Laplacian-PML preconditioner for the Helmholtz equation with perfectly matched layer (Helmholtz-PML equation). The Helmholtz-PML equation is discretized by an optimal 9-point difference scheme, and the preconditioned linear system is solved by the Krylov subspace method, especially by the biconjugate gradient stabilized method (Bi-CGSTAB). The spectral analysis of the linear system is given, and a new matrix-based interpolation operator is proposed for the multigrid method, which is used to approximately invert the preconditioner. The numerical experiments are presented to illustrate the efficiency of the preconditioned Bi-CGSTAB method with the multigrid based on the new interpolation operator, also, numerical results are given for comparing the performance of the new interpolation operator with that of classic bilinear interpolation operator and the one suggested in Erlangga et al. (2006) [10].     

A multigrid‐based shifted Laplacian preconditioner for a fourth‐order Helmholtz discretization

In this paper, an iterative solution method for a fourth‐order accurate discretization of the Helmholtz equation is presented. The method is a generalization of that presented in (SIAM J. Sci. Comput. 2006; 27 :1471–1492), where multigrid was employed as a preconditioner for a Krylov subspace iterative method. The multigrid preconditioner is based on the solution of a second Helmholtz operator with a complex‐valued shift. In particular, we compare preconditioners based on a point‐wise Jacobi smoother with those using an ILU(0) smoother, we compare using the prolongation operator developed by de Zeeuw in (J. Comput. Appl. Math. 1990; 33 :1–27) with interpolation operators based on algebraic multigrid principles, and we compare the performance of the Krylov subspace method Bi‐conjugate gradient stabilized with the recently introduced induced dimension reduction method, IDR(s). These three improvements are combined to yield an efficient solver for heterogeneous problems. Copyright © 2009 John Wiley & Sons, Ltd.     

Algebraic multigrid for complex symmetric matrices and applications

This paper is concerned with the numerical study of an algebraic multigrid preconditioner for complex symmetric system matrices. We use several different Krylov subspace methods as an outer iteration, namely the QMR method proposed by Freund and Nachtigal, the BiCGCR method of Clemens, and the CSYM method of Bunse-Gerstner and Stoever. In addition, we compare the results with the standard Jacobi preconditioner for complex symmetric problems. We test our approach on the numerical simulation of high-voltage insulators.     

几乎各向同性的高维空间分数阶扩散方程的分块快速正则Hermite分裂预处理方法

一类空间分数阶扩散方程经过有限差分离散后所得到的离散线性方程组的系数矩阵是两个对角矩阵与Toeplitz型矩阵的乘积之和.在本文中,对于几乎各向同性的二维或三维空间分数阶扩散方程的离散线性方程组,采用预处理Krylov子空间迭代方法,我们利用其系数矩阵的特殊结构和具体性质构造了一类分块快速正则Hermite分裂预处理子.通过理论分析,我们证明了所对应的预处理矩阵的特征值大部分都聚集于1的附近.数值实验也表明,这类分块快速正则Hermite分裂预处理子可以明显地加快广义极小残量(GMRES)方法和稳定化的双共轭梯度(BiCGSTAB)方法等Krylov子空间迭代方法的收敛速度.     

A new alternating positive semidefinite splitting preconditioner for saddle point problems from time-harmonic eddy current models

Based on the special positive semidefinite splittings of the saddle point matrix, we propose a new alternating positive semidefinite splitting (APSS) iteration method for the saddle point problem arising from the finite element discretization of the hybrid formulation of the time-harmonic eddy current problem. We prove that the new APSS iteration method is unconditionally convergent for both cases of the simple topology and the general topology. The new APSS matrix can be used as a preconditioner to accelerate the convergence rate of Krylov subspace methods. Numerical results show that the new APSS preconditioner is superior to the existing preconditioners.     

求解二维三温能量方程的基于AMG预条件子的Krylov子空间迭代法

本文对一类二维三温能量方程的实际应用问题，建立了一种半粗化的代数多重网格法（SAMG），进而得到了以该SAMG方法为预条件子的Krylov子空间迭代法。数值实验结果表明，该方法对求解二维三温能量方程的实际问题是十分有效和健壮的。     

Fast preconditioned iterative methods for finite volume discretization of steady-state space-fractional diffusion equations

We consider the preconditioned Krylov subspace method for linear systems arising from the finite volume discretization method of steady-state variable-coefficient conservative space-fractional diffusion equations. We propose to use a scaled-circulant preconditioner to deal with such Toeplitz-like discretization matrices. We show that the difference between the scaled-circulant preconditioner and the coefficient matrix is equal to the sum of a small-norm matrix and a low-rank matrix. Numerical tests are conducted to show the effectiveness of the proposed method for one- and two-dimensional steady-state space-fractional diffusion equations and demonstrate that the preconditioned Krylov subspace method converges very quickly.     

A review of formal orthogonality in Lanczos-based methods

Krylov subspace methods and their variants are presently the favorite iterative methods for solving a system of linear equations. Although it is a purely linear algebra problem, it can be tackled by the theory of formal orthogonal polynomials. This theory helps to understand the origin of the algorithms for the implementation of Krylov subspace methods and, moreover, the use of formal orthogonal polynomials brings a major simplification in the treatment of some numerical problems related to these algorithms. This paper reviews this approach in the case of Lanczos method and its variants, the novelty being the introduction of a preconditioner.     

A Projection Preconditioner for Solving the Implicit Immersed Boundary Equations

This paper presents a method for solving the linear semi-implicit immersed boundary equations which avoids the severe time step restriction presented by explicit-time methods. The Lagrangian variables are eliminated via a Schur complement to form a purely Eulerian saddle point system, which is preconditioned by a projection operator and then solved by a Krylov subspace method. From the viewpoint of projection methods, we derive an ideal preconditioner for the saddle point problem and compare the efficiency of a number of simpler preconditioners that approximate this perfect one. For low Reynolds number and high stiffness, one particular projection preconditioner yields an efficiency improvement of the explicit IB method by a factor around thirty. Substantial speed-ups over explicit-time method are achieved for Reynolds number below 100. This speedup increases as the Eulerian grid size and/or the Reynolds number are further reduced.     

On the Generalized Deteriorated Positive Semi-Definite and Skew-Hermitian Splitting Preconditioner

For nonsymmetric saddle point problems, Huang et al. in [Numer. Algor. 75 (2017), pp. 1161-1191] established a generalized variant of the deteriorated positive semi-definite and skew-Hermitian splitting (GVDPSS) preconditioner to expedite the convergence speed of the Krylov subspace iteration methods like the GMRES method. In this paper, some new convergence properties as well as some new numerical results are presented to validate the theoretical results.     

An efficient hierarchical preconditioner for quadratic discretizations of finite element problems

Higher order finite element discretizations, although providing higher accuracy, are considered to be computationally expensive and of limited use for large‐scale problems. In this paper, we have developed an efficient iterative solver for solving large‐scale quadratic finite element problems. The proposed approach shares some common features with geometric multigrid methods but does not need structured grids to create the coarse problem. This leads to a robust method applicable to finite element problems discretized by unstructured meshes such as those from adaptive remeshing strategies. The method is based on specific properties of hierarchical quadratic bases. It can be combined with an algebraic multigrid (AMG) preconditioner or with other algebraic multilevel block factorizations. The algorithm can be accelerated by flexible Krylov subspace methods. We present some numerical results on the convection–diffusion and linear elasticity problems to illustrate the efficiency and the robustness of the presented algorithm. In these experiments, the performance of the proposed method is compared with that of an AMG preconditioner and other iterative solvers. Our approach requires less computing time and less memory storage. Copyright © 2010 John Wiley & Sons, Ltd.     

Comparative study on order-reduced methods for linear third-order ordinary differential equations

The linear third-order ordinary differential equation (ODE) can be transformed into a system of two second-order ODEs by introducing a variable replacement, which is different from the common order-reduced approach. We choose the functions p(x) and q(x) in the variable replacement to get different cases of the special order-reduced system for the linear third-order ODE. We analyze the numerical behavior and algebraic properties of the systems of linear equations resulting from the sinc discretizations of these special second-order ODE systems. Then the block-diagonal preconditioner is used to accelerate the convergence of the Krylov subspace iteration methods for solving the discretized system of linear equation. Numerical results show that these order-reduced methods are effective for solving the linear third-order ODEs.     

Approximate inverse preconditioner by computing approximate solution of Sylvester equation

This paper presents a new method for obtaining a matrix M which is an approximate inverse preconditioner for a given matrix A, where the eigenvalues of A all either have negative real parts or all have positive real parts. This method is based on the approximate solution of the special Sylvester equation AX + XA = 2I. We use a Krylov subspace method for obtaining an approximate solution of this Sylvester matrix equation which is based on the Arnoldi algorithm and on an integral formula. The computation of the preconditioner can be carried out in parallel and its implementation requires only the solution of very simple and small Sylvester equations. The sparsity of the preconditioner is preserved by using a proper dropping strategy. Some numerical experiments on test matrices from Harwell–Boing collection for comparing the numerical performance of the new method with an available well-known algorithm are presented.     

A modified positive-definite and skew-Hermitian splitting preconditioner for generalized saddle point problems from the Navier-Stokes equation

In this paper, we extend the relaxed positive-definite and skew-Hermitian splitting preconditioner (RPSS) for generalized saddle-point problems in [J.-L. Zhang, C.-Q. Gu and K. Zhang, Appl. Math. Comput. 249(2014)468-479] by introducing an additional parameter. The spectral properties of the presented new preconditioned matrix for generalized saddle-point problem are investigated, meanwhile, the infinite termination merit of the iterative step is also discussed if the Krylov subspace method preconditioned by the modified positive-definite and skew-Hermitian splitting preconditioner (MPSS) is applied. Some numerical experiments illustrate that the efficiency of the proposed new preconditioner.     

广义鞍点问题的松弛维数分解预条件子

本文将Benzi等提出的松弛维数分解(Relaxed dimensionalfactorization, RDF)预条件子进一步推广到广义鞍点问题上,并称为GRDF(Generalized RDF)预条件子.该预条件子可看做是用维数分裂迭代法求解广义鞍点问题而导出的改进维数分裂(Modified dimensional split, MDS)预条件子的松弛形式, 它相比MDS预条件子更接近于系数矩阵, 因而结合Krylov子空间方法(如GMRES)有更快的收敛速度.文中分析了GRDF预处理矩阵特征值的一些性质,并用数值算例验证了新预条件子的有效性.     

On a type of matrix splitting preconditioners for a class of block two-by-two linear systems

Recently, Bai et al. (2013) proposed an effective and efficient matrix splitting iterative method, called preconditioned modified Hermitian/skew-Hermitian splitting (PMHSS) iteration method, for two-by-two block linear systems of equations. The eigenvalue distribution of the iterative matrix suggests that the splitting matrix could be advantageously used as a preconditioner. In this study, the CGNR method is utilized for solving the PMHSS preconditioned linear systems, and the performance of the method is considered by estimating the condition number of the normal equations. Furthermore, the proposed method is compared with other PMHSS preconditioned Krylov subspace methods by solving linear systems arising in complex partial differential equations and a distributed control problem. The numerical results demonstrate the difference in the performance of the methods under consideration.     

Hierarchical Preconditioning for PT AP-Systems arising from particulate flows

Particulate flows, i.e. flow of an incompressible, Newtonian carrier fluid loaded with (many) rigid bodies, play an important role in diverse technical applications. In [1] a finite element method was introduced to simulate such flows. The method relies on a splitting method and a subspace projection method to incorporate the rigid body motion of the particles in a so called one domain approach [2]. The resulting systems arising after discretization are ill conditioned in general. Thus preconditioning is mandatory for instance when using an iterative Krylov subspace methods. In this paper we present a Schur complement based preconditioner well suited for this type of application. (© 2012 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

A nearly optimal preconditioner for the Navier–Stokes equations

We present a preconditioner for the linearized Navier–Stokes equations which is based on the combination of a fast transform approximation of an advection diffusion problem together with the recently introduced ‘BFBT^T’ preconditioner of Elman (SIAM Journal of Scientific Computing, 1999; 20 :1299–1316). The resulting preconditioner when combined with an appropriate Krylov subspace iteration method yields the solution in a number of iterations which appears to be independent of the Reynolds number provided a mesh Péclet number restriction holds, and depends only mildly on the mesh size. The preconditioner is particularly appropriate for problems involving a primary flow direction. Copyright © 2001 John Wiley & Sons, Ltd.