Reordering and incomplete preconditioning in serial and parallel adaptive mesh refinement and coarsening flow solutions

The effects of reordering the unknowns on the convergence of incomplete factorization preconditioned Krylov subspace methods are investigated. Of particular interest is the resulting preconditioned iterative solver behavior when adaptive mesh refinement and coarsening (AMR/C) are utilized for serial or distributed parallel simulations. As representative schemes, we consider the familiar reverse Cuthill–McKee and quotient minimum degree algorithms applied with incomplete factorization preconditioners to CG and GMRES solvers. In the parallel distributed case, reordering is applied to local subdomains for block ILU preconditioning, and subdomains are repartitioned dynamically as mesh adaptation proceeds. Numerical studies for representative applications are conducted using the object‐oriented AMR/C software system libMesh linked to the PETSc solver library. Serial tests demonstrate that global unknown reordering and incomplete factorization preconditioning can reduce the number of iterations and improve serial CPU time in AMR/C computations. Parallel experiments indicate that local reordering for subdomain block preconditioning associated with dynamic repartitioning because of AMR/C leads to an overall reduction in processing time. Copyright © 2011 John Wiley & Sons, Ltd.     

A Newton–Krylov finite volume algorithm for the power-law non-Newtonian fluid flow using pseudo-compressibility technique

An implicit Newton–Krylov finite volume algorithm has been developed for efficient steady-state computation of the power-law non-Newtonian fluid flows. The pseudo-compressibility technique is used for the coupling of continuity and momentum equations. The spatial discretization is central (second-order) for both convective and diffusive terms and the accuracy of the solution is verified. The nine block diagonal Jacobian matrix (needed for implicit formulation) is computed directly through the flux differentiation. Five-diagonal and three-diagonal block matrices (the simplified versions of the main Jacobian matrix) are used with the ILU(0 & 1) and the Thomas linear solvers for preconditioning, respectively. The performance of the Newton-GMRES solver is examined in detail for different preconditioning strategies. The effects of the power-law behavior index and Re number on the convergence rate are also studied. The performance of the Newton-BiCGSTAB and the Newton-GMRES solvers are compared with each other. The results show, the ILU(1)/Newton-GMRES is the most efficient combination that is robust even in high Reynolds number shear-thinning fluid flow cases.     

二维定常湍流计算中的GMRES算法

在以前工作的基础上将广义极小残差（Generalized Minimum RESidual (GMRES)算法发展到用于求解二维可压Favier平均Navier-Stokes方程组。控制方程经Newton线化处理后构成近似的线性系统,然后采用分别耦合了LUSGS和ILU两种预处理矩阵的GMRES算法求解。Spalart-Allmaras湍流模型被用来封闭流体控制方程组,采用与流体控制方程非耦合的方式,使用LUSGS方法求解。对GMRES算法中矩阵－向量的乘积采用了有限插分方法,从而避免了精确的左端系数矩阵的计算和存储。对预处理矩阵的两种使用方法（左预处理和右预处理）进行了分析和讨论。用两个算例对LUSGS和ILU两各预处理矩阵进行了比较,同时探讨了左预处理和右预处理各自的优缺点。通过对Sajben扩压器和NACA0012有攻角流动的计算,表明带有预处理的GMRES算法在二维定常跨音黏性流动计算中相比于得到广泛应用的DDADI方法具有很大优势,左预处理要优于右预处理。     

Parallel ILU preconditioning,a priori pivoting and segregation of variables for iterative solution of the mixed finite element formulation of the Navier–Stokes equations

A parallel ILU preconditioning algorithm for the incompressible Navier–Stokes equations has been designed, implemented and tested. The computational mesh is divided into N subdomains which are processed in parallel in different processors. During ILU factorization, matrices and vectors associated with the nodes on the interface between the subdomains are communicated to the equation matrices to the adjacent subdomain. The bases for the parallel algorithm are an appropriate node ordering scheme and a segregation of velocity and pressure degrees of freedom. The inner nodes of the subdomain are numbered first and then the nodes on the interface between the subdomains. To avoid division by zero during the ILU factorization, the equations corresponding to the velocity degrees of freedom are assembled first in the global equation matrix, followed by the equations corresponding to the pressure degrees of freedom. Copyright © 2003 John Wiley & Sons, Ltd.     

A parallel finite element sliding mesh technique for the simulation of viscous flows in agitated tanks

A parallel sliding mesh algorithm for the finite element simulation of viscous fluid flows in agitated tanks is presented. Lagrange multipliers are used at the sliding interfaces to enforce the continuity between the fixed and moving subdomains. The novelty of the method consists of the coupled solution of the resulting velocity–pressure‐Lagrange multipliers system of equations by an ILU(0)‐QMR solver. A penalty parameter is introduced for both the interface and the incompressibility constraints to avoid pivoting problems in the ILU(0) algorithm. To handle the convective term, both the Newton–Raphson scheme and the semi‐implicit linearization are tested. A penalty parameter is introduced for both the interface and the incompressibility constraints to avoid the failure of the ILU(0) algorithm due to the lack of pivoting. Furthermore, this approach is versatile enough so that it allows partitioning of sliding and fixed subdomains if parallelization is required. Although the sliding mesh technique is fairly common in CFD, the main advantage of the proposed approach is its low computational cost due to the inexpensive and parallelizable calculations that involve preconditioned sparse iterative solvers. The method is validated for Couette and coaxial stirred tanks. Copyright © 2011 John Wiley & Sons, Ltd.     

Numerical solutions of fully non‐linear and highly dispersive Boussinesq equations in two horizontal dimensions

This paper investigates preconditioned iterative techniques for finite difference solutions of a high‐order Boussinesq method for modelling water waves in two horizontal dimensions. The Boussinesq method solves simultaneously for all three components of velocity at an arbitrary z‐level, removing any practical limitations based on the relative water depth. High‐order finite difference approximations are shown to be more efficient than low‐order approximations (for a given accuracy), despite the additional overhead. The resultant system of equations requires that a sparse, unsymmetric, and often ill‐conditioned matrix be solved at each stage evaluation within a simulation. Various preconditioning strategies are investigated, including full factorizations of the linearized matrix, ILU factorizations, a matrix‐free (Fourier space) method, and an approximate Schur complement approach. A detailed comparison of the methods is given for both rotational and irrotational formulations, and the strengths and limitations of each are discussed. Mesh‐independent convergence is demonstrated with many of the preconditioners for solutions of the irrotational formulation, and solutions using the Fourier space and approximate Schur complement preconditioners are shown to require an overall computational effort that scales linearly with problem size (for large problems). Calculations on a variable depth problem are also compared to experimental data, highlighting the accuracy of the model. Through combined physical and mathematical insight effective preconditioned iterative solutions are achieved for the full physical application range of the model. Copyright © 2004 John Wiley & Sons, Ltd.     

GPU‐accelerated direct numerical simulations of decaying compressible turbulence employing a GKM‐based solver

Gas Kinetic Method‐based flow solvers have become popular in recent years owing to their robustness in simulating high Mach number compressible flows. We evaluate the performance of the newly developed analytical gas kinetic method (AGKM) by Xuan et al. in performing direct numerical simulation of canonical compressible turbulent flow on graphical processing unit (GPU)s. We find that for a range of turbulent Mach numbers, AGKM results shows excellent agreement with high order accurate results obtained with traditional Navier–Stokes solvers in terms of key turbulence statistics. Further, AGKM is found to be more efficient as compared with the traditional gas kinetic method for GPU implementation. We present a brief overview of the optimizations performed on NVIDIA K20 GPU and show that GPU optimizations boost the speedup up‐to 40x as compared with single core CPU computations. Hence, AGKM can be used as an efficient method for performing fast and accurate direct numerical simulations of compressible turbulent flows on simple GPU‐based workstations. Copyright © 2016 John Wiley & Sons, Ltd.     

Practical aspects of p‐multigrid discontinuous Galerkin solver for steady and unsteady RANS simulations

Efficient and robust p‐multigrid solvers are presented for solving the system arising from high‐order discontinuous Galerkin discretizations of the compressible Reynolds‐Averaged Navier–Stokes (RANS) equations. Two types of multigrid methods and a multigrid preconditioned Newton–Krylov method are investigated, and both steady and unsteady algorithms are considered in this paper. For steady algorithms, a new strategy is introduced to determine the CFL number, which has been proved to be critical in achieving the effective and stable convergence for p‐multigrid methods. We also suggest a modified smoothing technique to further improve the efficiency of the algorithms. For unsteady algorithms, special attention has been paid to the cycling strategy and the full multigrid technique, and we point out a significant difference on the parameter selection for unsteady computations. The capabilities of the resulted solvers have been examined by performing steady and unsteady RANS simulations. Comparative assessment in terms of efficiency, robustness, and memory consumption are carried out for all solvers. Copyright © 2015 John Wiley & Sons, Ltd.     

On the efficiency and quality of numerical solutions in CFD problems using the interface strip preconditioner for domain decomposition methods

In this paper, some pathologies found for simple tests solved by means of preconditioned full iterative schemes are presented. According to these results (Sections 4 and 5), the accuracy deterioration observed should be considered as a warning for the final application given to these solutions. Even though it is well known that full iterative solvers are not the best selection for comparison, they were chosen because they are widely used by the computational fluid dynamic (CFD) community for a diversity of complex fluid dynamics applications. FEM simulated solutions are compared with analytical solutions or measured data for problems that have been considered as ‘benchmarks’ in the CFD literature. For this purpose, the study of the solution obtained via parallelized iterative methods that have been extensively used (e.g. conjugate gradients (CG), GMRes global iteration and its variants, ‘overlapping’ and ‘non‐overlapping’ additive Schwarz domain decomposition schemes) in CFD computations and those obtained with the new interface strip preconditioner (J. Comput. Meth. Sci. Engng 2003; Int. J. Numer. Meth. Engng 2005; 62 (13):1873–1894) for the Schur complement method is carried out. The idea is to present the new solver as an alternative to obtain more accurate and faster solutions in the context of monolithic and non‐monolithic schemes applied to a internal/external viscous compressible/incompressible flows around bodies of complex shapes. Therefore, the target of this work is to show how the reliability of CFD codes is affected by the solver selection and why domain decomposition methods should be viewed not only as a more efficient strategy, but also to guarantee the solution quality. Copyright © 2006 John Wiley & Sons, Ltd.     

Segregated finite element algorithms for the numerical solution of large-scale incompressible flow problems

This paper presents results of an ongoing research program directed towards developing fast and efficient finite element solution algorithms for the simulation of large-scale flow problems. Two main steps were taken towards achieving this goal. The first step was to employ segregated solution schemes as opposed to the fully coupled solution approach traditionally used in many finite element solution algorithms. The second step was to replace the direct Gaussian elimination linear equation solvers used in the first step with iterative solvers of the conjugate gradient and conjugate residual type. The three segregated solution algorithms developed in step one are first presented and their integrity and relative performance demonstrated by way of a few examples. Next, the four types of iterative solvers (i.e. two options for solving the symmetric pressure type equations and two options for solving the non-symmetric advection–diffusion type equations resulting from the segregated algorithms) together with the two preconditioning strategies employed in our study are presented. Finally, using examples of practical relevance the paper documents the large gains which result in computational efficiency, over fully coupled solution algorithms, as each of the above two main steps are introduced. It is shown that these gains become increasingly more dramatic as the complexity and size of the problem is increased.     

Parallelization of the ILU(0) preconditioner for CFD problems on shared‐memory computers

The use of ILU(0) factorization as a preconditioner is quite frequent when solving linear systems of CFD computations. This is because of its efficiency and moderate memory requirements. For a small number of processors, this preconditioner, parallelized through coloring methods, shows little savings when compared with a sequential one using adequate reordering of the unknowns. Level scheduling techniques are applied to obtain the same preconditioning efficiency as in a sequential case, while taking advantage of parallelism through block algorithms. Numerical results obtained from the parallel solution of the compressible Navier–Stokes equations show that this technique gives interesting savings in computational times on a small number of processors of shared‐memory computers. In addition, it does this while keeping all the benefits of an ILU(0) factorization with an adequate reordering of the unknowns, and without the loss of efficiency of factorization associated with a more scalable coloring strategy. Copyright © 1999 John Wiley & Sons, Ltd.     

Massively parallel mesh adaptation and linear system solution for multiphase flows

ABSTRACT

In this paper, a work performed to allow massively parallel finite element flow computations is presented. It includes the development and optimisation of two particular features of a finite element multiphase computational fluid dynamics software, which are mesh generation and linear system solution, using anisotropic adaptation and multigrid preconditioning. Parallel performances on supercomputers are shown, where the largest generated mesh (on 65 536 Intel Xeon or 261 144 Power PC cores) had 33.4 billions of nodes, leading to a 100 billion of unknowns linear system solution. Final applications concern, between others, image-based flow simulations.     

Iterative solvers for quadratic discretizations of the generalized Stokes problem

We present in this paper various iterative methods for the solution of large linear and non‐linear systems resulting from the discretization of the generalized Stokes problem. A second‐order (O(h²)) P₂‐P₁ mixed finite element is used for the approximation of the velocity and the pressure. Solution strategies based on conjugate gradient‐like methods, the Uzawa's and Arrow–Hurwicz's methods are presented. Schur complement methods are also explored in the context of a hierarchical decomposition of the velocity field. The ever present preconditioning problem is also addressed. The performance of these iterative methods will be discussed on complex flows of industrial interest. Copyright © 2004 John Wiley & Sons, Ltd.     

An ILU preconditioner with coupled node fill-in for iterative solution of the mixed finite element formulation of the 2D and 3D Navier-Stokes equations

In the present paper, preconditioning of iterative equation solvers for the Navier-Stokes equations is investigated. The Navier-Stokes equations are solved for the mixed finite element formulation. The linear equation solvers used are the orthomin and the Bi-CGSTAB algorithms. The storage structure of the equation matrix is given special attention in order to avoid swapping and thereby increase the speed of the preconditioner. The preconditioners considered are Jacobian, SSOR and incomplete LU preconditioning of the matrix associated with the velocities. A new incomplete LU preconditioning with fill-in for the pressure matrix at locations in the matrix where the corner nodes are coupled is designed. For all preconditioners, inner iterations are investigated for possible improvement of the preconditioning. Numerical experiments are executed both in two and three dimensions.     

Iterative-method performance evaluation for multiple vectors associated with a large-scale sparse matrix

ABSTRACT

Ensemble computing, which is an instance of capacity computing, is an effective computing scenario for exascale parallel supercomputers. In ensemble computing, there are multiple linear systems associated with a common coefficient matrix. We improve the performance of iterative solvers for multiple vectors by solving them at the same time, that is, by solving for the product of the matrices. We implemented several iterative methods and compared their performance. The maximum performance on Sparc VIIIfx was 7.6 times higher than that of a naïve implementation. Finally, to deal with the different convergence processes of linear systems, we introduced a control method to eliminate the calculation of already converged vectors.     

A numerical method for 3D barotropic flows in turbomachinery

A numerical method for the simulation of 3D inviscid barotropic flows in rotating frames is presented. A barotropic state law incorporating a homogeneous-flow cavitation model is considered. The discretisation is based on a finite-volume formulation applicable to unstructured grids. A shock-capturing Roe-type upwind scheme is proposed for barotropic flows. The accuracy of the proposed method at low Mach numbers is ensured by ad-hoc preconditioning, preserving time consistency. An implicit time advancing only relying on the algebraic properties of the Roe flux function, and thus applicable to a variety of problems, is presented. The proposed numerical ingredients, already validated in a 1D context and applied to 3D non-rotating computations, are then applied to the 3D water flow around a typical turbopump inducer.     

Preconditioned Krylov subspace methods used in solving two‐dimensional transient two‐phase flows

This paper investigates the performance of preconditioned Krylov subspace methods used in a previously presented two‐fluid model developed for the simulation of separated and intermittent gas–liquid flows. The two‐fluid model has momentum and mass balances for each phase. The equations comprising this model are solved numerically by applying a two‐step semi‐implicit time integration procedure. A finite difference numerical scheme with a staggered mesh is used. Previously, the resulting linear algebraic equations were solved by a Gaussian band solver. In this study, these algebraic equations are also solved using the generalized minimum residual (GMRES) and the biconjugate gradient stabilized (Bi‐CGSTAB) Krylov subspace iterative methods preconditioned with incomplete LU factorization using the ILUT(p, τ) algorithm. The decrease in the computational time using the iterative solvers instead of the Gaussian band solver is shown to be considerable. Copyright © 1999 John Wiley & Sons, Ltd.     

Investigating cross-wind stability of high-speed trains with large-scale parallel CFD

ABSTRACT

The side-wind loading on a simplified train model at scale 1:25 is investigated by parallel large eddy simulation (LES) with incompressible solvers from the OpenFOAM package and a novel dynamically adaptive, parallel LES-type lattice Boltzmann method (LBM) implemented in our own AMROC framework. It is found that the new LBM code provides more accurate time-averaged force predictions, while compute times are reduced.     

A fast and efficient iterative scheme for viscoelastic flow simulations with the DEVSS finite element method

We present a new fast iterative solution technique for the large sparse-matrix system that is commonly encountered in the mixed finite-element formulation of transient viscoelastic flow simulation: the DEVSS (discrete elastic-viscous stress splitting) method. A block-structured preconditioner for the velocity, pressure and viscous polymer stress has been proposed, based on a block reduction of the discrete system, designed to maintain spectral equivalence with the discrete system. The algebraic multigrid method and the diagonally scaled conjugate gradient method are applied to the preconditioning sub-block systems and a Krylov subspace iterative method (MINRES) is employed as an outer solver. We report the performance of the present solver through example problems in 2D and 3D, in comparison with the corresponding Stokes problems, and demonstrate that the outer iteration, as well as each block preconditioning sub-problem, can be solved within a fixed number of iterations. The required CPU time for the entire problem scales linearly with the number of degrees of freedom, indicating O(N) performance of this solution algorithm.