The implementation of an aeronautical CFD flow code onto distributed memory parallel systems

The parallelization of an industrially important in‐house computational fluid dynamics (CFD) code for calculating the airflow over complex aircraft configurations using the Euler or Navier–Stokes equations is presented. The code discussed is the flow solver module of the SAUNA CFD suite. This suite uses a novel grid system that may include block‐structured hexahedral or pyramidal grids, unstructured tetrahedral grids or a hybrid combination of both. To assist in the rapid convergence to a solution, a number of convergence acceleration techniques are employed including implicit residual smoothing and a multigrid full approximation storage scheme (FAS). Key features of the parallelization approach are the use of domain decomposition and encapsulated message passing to enable the execution in parallel using a single programme multiple data (SPMD) paradigm. In the case where a hybrid grid is used, a unified grid partitioning scheme is employed to define the decomposition of the mesh. The parallel code has been tested using both structured and hybrid grids on a number of different distributed memory parallel systems and is now routinely used to perform industrial scale aeronautical simulations. Copyright © 2000 John Wiley & Sons, Ltd.     

A PARALLEL BLOCK-STRUCTURED MULTIGRID METHOD FOR THE PREDICTION OF INCOMPRESSIBLE FLOWS

In this paper a parallel multigrid finite volume solver for the prediction of steady and unsteady flows in complex geometries is presented. For the handling of the complexity of the geometry and for the parallelization a unified approach connected with the concept of block-structured grids is employed. The parallel implementation is based on grid partitioning with automatic load balancing and follows the message-passing concept, ensuring a high degree of portability. A high numerical efficiency is obtained by a non-linear multigrid method with a pressure correction scheme as smoother. By a number of numerical experiments on various parallel computers the method is investigated with respect to its numerical and parallel efficiency. The results illustrate that the high performance of the underlying sequential multigrid algorithm can largely be retained in the parallel implementation and that the proposed method is well suited for solving complex flow problems on parallel computers with high efficiency.     

ADAPTIVE PARALLEL MULTIGRID SOLUTION OF 2D INCOMPRESSIBLE NAVIER–STOKES EQUATIONS

In this paper an adaptive parallel multigrid method and an application example for the 2D incompressible Navier–Stokes equations are described. The strategy of the adaptivity in the sense of local grid refinement in the multigrid context is the multilevel adaptive technique (MLAT) suggested by Brandt. The parallelization of this method on scalable parallel systems is based on the portable communication library CLIC and the message-passing standards: PARMACS, PVM and MPI. The specific problem considered in this work is a two-dimensional hole pressure problem in which a Poiseuille channel flow is disturbed by a cavity on one side of the channel. Near geometric singularities a very fine grid is needed for obtaining an accurate solution of the pressure value. Two important issues of the efficiency of adaptive parallel multigrid algorithms, namely the data redistribution strategy and the refinement criterion, are discussed here. For approximate dynamic load balancing, new data in the adaptive steps are redistributed into distributed memories in different processors of the parallel system by block remapping. Among several refinement criteria tested in this work, the most suitable one for the specific problem is that based on finite-element residuals from the point of view of self-adaptivity and computational efficiency, since it is a kind of error indicator and can stop refinement algorithms in a natural way for a given tolerance. Comparisons between different global grids without and with local refinement have shown the advantages of the self-adaptive technique, as this can save computer memory and speed up the computing time several times without impairing the numerical accuracy. © 1997 By John Wiley & Sons, Ltd. Int. J. Numer. Methods Fluids 24, 875–892, 1997.     

建筑风压数值模拟的几种并行化策略

针对四个处理机的Ｔｒａｎｓｐｕｔｅｒ并行计算机系统，建立了建筑风压数值模拟问题基于ＳＩＭ－ＰＬＥＣ算法的几种并行化策略：分区并行策略、方程并行策略和双重并行策略。对各种策略的计算流程、数据通讯及并行效率等进行了分析和比较，并通过实例计算作了验证。     

A comparison of coarse and fine grain parallelization strategies for the simple pressure correction algorithm

The primary aim of this work was to determine the simplest and most effective parallelization strategy for control-volume-based codes solving industrial problems. It has been found that for certain classes of problems, the coarse-grain functional decomposition strategy, largely ignored due to its limited scaling capability, offers the potential for significant execution speed-ups while maintaining the inherent structure of traditional serial algorithms. Functional decomposition requires only minor modification of the existing serial code to implement and, hence, code portability across both concurrent and serial computers is maintained. Fine-grain parallelization strategies at the ‘DO loop’ level are also easy to implement and largely preserve code portability. Both coarse-grain functional decomposition and fine-grain loop-level parallelization strategies for the SIMPLE pressure correction algorithm are demonstrated on a Silicon Graphics 4D280S eight CPU shared memory computer system for a highly coupled, transient two-dimensional simulation involving melting of a metal in the presence of thermal-buoyancy-driven laminar convection. Problems requiring the solution of a larger number of transport equations were simulated by including further scalar variables in the calculation. While resulting in slight degradation of the convergence rate, the functional decomposition strategy exhibited higher parallel efficiencies and yielded greater speed-ups relative to the original serial code. Initially, this strategy showed a significant degradation in convergence rate due to an inconsistency in the parallel solution of the pressure correction equation. After correcting for this inconsistency, the maximum speed-up for 16 dependent variables was a factor of 5·28 with eight processors, representing a parallel efficiency of 67%. Peak efficiency of 76% was achieved using five processors to solve for 10 dependent variables.     

THE PARALLEL BLOCK ADAPTIVE MULTIGRID METHOD FOR THE IMPLICIT SOLUTION OF THE EULER EQUATIONS

A method capable of solving very fast and robust complex non-linear systems of equations is presented. The block adaptive multigrid (BAM) method combines mesh adaptive techniques with multigrid and domain decomposition methods. The overall method is based on the FAS multigrid, but instead of using global grids, locally enriched subgrids are also employed in regions where excessive solution errors are encountered. The final mesh is a composite grid with uniform rectangular subgrids of various mesh densities. The regions where finer grid resolution is necessary are detected using an estimation of the solution error by comparing solutions between grid levels. Furthermore, an alternative domain decomposition strategy has been developed to take advantage of parallel computing machines. The proposed method has been applied to an implicit upwind Euler code (EuFlex) for the solution of complex transonic flows around aerofoils. The efficiency and robustness of the BAM method are demonstrated for two popular inviscid test cases. Up to 19-fold acceleration with respect to the single-grid solution has been achieved, but a further twofold speed-up is possible on four-processor parallel computers.     

Vectorized Directional Sweep Based Parallelization Method for Factored Alternating Direction Implicit (ADI) Schemes

A new parallelization method is proposed for factored alternating direction implicit (ADI) schemes based on the vectorized global domain directional sweep. This approach, when combined with multi-partitioning domain decomposition, significantly reduces the frequency of necessary communication calls and minimizes processor idling during the sweeping processes. The present parallelization approach is applied to a number of vectorized two-dimensional compressible Navier-Stokes solvers. The codes vary in complexity from laminar to algebraic turbulence closure model and finally the strongly coupled Navier-Stokes and K-e equations. Implementation is conducted using PVM (Parallel Virtual Machine) message passing tool on the Cray T3D massively parallel processing (MPP) machine. The implemented parallel codes are assessed in terms of accuracy and parallel performance.     

Multigrid flow solutions in complex two-dimensional geometries

A finite difference solution algorithm is described for use on two-dimensional curvilinear meshes generated by the solution of the transformed Laplace equation. The efficiency of the algorithm is improved through the use of a full approximation scheme (FAS) multigrid algorithm using an extended pressure correction scheme as smoother. The multigrid algorithm is implemented as a fixed V-cycle through the grid levels with a constant number of sweeps being performed at each grid level. The accuracy and efficiency of the numerical code are validated using comparisons of the flow over two backward step configurations. Results show close agreement with previous numerical predictions and experimental data. Using a standard Cartesian co-ordinate flow solver, the multigrid efficiency obtainable in a rectangular system is shown to be reproducible in two-dimensional body-fitted curvilinear co-ordinates. Comparisons with a standard one-grid method show the multigrid method, on curvilinear meshes, to give reductions in CPU time of up to 93%.     

Parallel solution of lifting rotors in hover and forward flight

An implicit unsteady, multiblock, multigrid, upwind solver including mesh deformation capability, and structured multiblock grid generator, are presented and applied to lifting rotors in both hover and forward flight. To allow the use of very fine meshes and, hence, better representation of the flow physics, a parallel version of the code has been developed. It is demonstrated that once the grid density is sufficient to capture enough turns of the tip vortices, hover exhibits oscillatory behaviour of the wake, even using a steady formulation. An unsteady simulation is then presented, and detailed analysis of the time‐accurate wake history is performed and compared to theoretical predictions. Forward flight simulations are also presented and, again, grid density effects on the wake formation investigated. Parallel performance of the code using up to 1024 CPU's is also presented. Copyright © 2006 John Wiley & Sons, Ltd.     

Parallel implementation of a non‐hydrostatic model for free surface flows with semi‐Lagrangian advection treatment

The parallel implementation of an unstructured‐grid, three‐dimensional, semi‐implicit finite difference and finite volume model for the free surface Navier–Stokes equations (UnTRIM ) is presented and discussed. The new developments are aimed to make the code available for high‐performance computing in order to address larger, complex problems in environmental free surface flows. The parallelization is based on the mesh partitioning method and message passing and has been achieved without negatively affecting any of the advantageous properties of the serial code, such as its robustness, accuracy and efficiency. The key issue is a new, autonomous parallel streamline backtracking algorithm, which allows using semi‐Lagrangian methods in decomposed meshes without compromising the scalability of the code. The implementation has been carefully verified not only with simple, abstract test cases illustrating the application domain of the code but also with advanced, high‐resolution models presently applied for research and engineering projects. The scheme performance and accuracy aspects are researched and discussed. Copyright © 2008 John Wiley & Sons, Ltd.     

A Discrete Adjoint Approach for the Optimization of Unsteady Turbulent Flows

In this paper we present a discrete adjoint approach for the optimization of unsteady, turbulent flows. While discrete adjoint methods usually rely on the use of the reverse mode of Automatic Differentiation (AD), which is difficult to apply to complex unsteady problems, our approach is based on the discrete adjoint equation directly and can be implemented efficiently with the use of a sparse forward mode of AD. We demonstrate the approach on the basis of a parallel, multigrid flow solver that incorporates various turbulence models. Due to grid deformation routines also shape optimization problems can be handled. We consider the relevant aspects, in particular the efficient generation of the discrete adjoint equation and the parallel implementation of a multigrid method for the adjoint, which is derived from the multigrid scheme of the flow solver. Numerical results show the efficiency of the approach for a shape optimization problem involving a three dimensional Large Eddy Simulation (LES).     

A methodology for high performance computation of fully inhomogeneous turbulent flows

A large‐eddy simulation methodology for high performance parallel computation of statistically fully inhomogeneous turbulent flows on structured grids is presented. Strategies and algorithms to improve the memory efficiency as well as the parallel performance of the subgrid‐scale model, the factored scheme, and the Poisson solver on shared‐memory parallel platforms are proposed and evaluated. A novel combination of one‐dimensional red–black/line Gauss–Seidel and two‐dimensional red–black/line Gauss–Seidel methods is shown to provide high efficiency and performance for multigrid relaxation of the Poisson equation. Parallel speedups are measured on various shared‐distributed memory systems. Validations of the code are performed in large‐eddy simulations of turbulent flows through a straight channel and a square duct. Results obtained from the present solver employing a Lagrangian dynamic subgrid‐scale model show good agreements with other available data. The capability of the code for more complex flows is assessed by performing a large‐eddy simulation of the tip‐leakage flow in a linear cascade. Copyright © 2006 John Wiley & Sons, Ltd.     

A component framework for the parallel solution of the incompressible Navier–Stokes equations with Radau‐IIA methods

An efficient solution strategy for the simulation of incompressible fluids needs adequate and accurate space and time discretization schemes. In this paper, for the space discretization, we use an inf–sup stable finite element method and for the time discretization, Radau‐IIA methods of higher order, which have the advantage that the pressure component has convergence order s in time, where s is the number of internal stages. The disadvantage of this approach is that we have a high computational amount of work, because large nonlinear systems of equations have to solved. In this paper, we use a transformation of the coefficient matrix and the simplified Newton method. This approach has the effect that our large nonlinear systems split into smaller ones, which can now also be solved in parallel. For the parallelization of the code we use the software component technology and the Component Template Library. Numerical examples show that high order in the pressure component can be achieved and that the proposed solution technique is very effective. Copyright © 2015 John Wiley & Sons, Ltd.     

结构静力有限元分层并行计算方法

根据分布式存储并行计算机的体系结构特点,提出了一种结构静力有限元分层并行计算方法. 该方法在两级分区两次缩聚策略的基础上不仅实现了大量数据的分布式存储,提高了数据的内存访问速率;而且实现了计算过程的三层并行,有效提高了通信效率;此外,它还进一步降低了界面方程的规模,大幅度减少了界面方程的求解时间. 因此,它能够充分利用分布式存储并行计算机的体系结构特点提升大规模并行计算效率. 最后通过典型数值算例验证了该方法的正确性和有效性.      

Implicit symmetrized streamfunction formulations of magnetohydrodynamics

We apply the finite element method to the classic tilt instability problem of two‐dimensional, incompressible magnetohydrodynamics, using a streamfunction approach to enforce the divergence‐free conditions on the magnetic and velocity fields. We compare two formulations of the governing equations, the standard one based on streamfunctions and a hybrid formulation with velocities and magnetic field components. We use a finite element discretization on unstructured meshes and an implicit time discretization scheme. We use the PETSc library with index sets for parallelization. To solve the nonlinear problems on each time step, we compare two nonlinear Gauss‐Seidel‐type methods and Newton's method with several time‐step sizes. We use GMRES in PETSc with multigrid preconditioning to solve the linear subproblems within the nonlinear solvers. We also study the scalability of this simulation on a cluster. Copyright © 2008 John Wiley & Sons, Ltd.     

A HIERARCHICAL PARALLEL COMPUTING APPROACH FOR STRUCTURAL STATIC FINITE ELEMENT ANALYSIS

根据分布式存储并行计算机的体系结构特点,提出了一种结构静力有限元分层并行计算方法. 该方法在两级分区两次缩聚策略的基础上不仅实现了大量数据的分布式存储,提高了数据的内存访问速率;而且实现了计算过程的三层并行,有效提高了通信效率;此外,它还进一步降低了界面方程的规模,大幅度减少了界面方程的求解时间. 因此,它能够充分利用分布式存储并行计算机的体系结构特点提升大规模并行计算效率. 最后通过典型数值算例验证了该方法的正确性和有效性.     

Parallel large eddy simulation of turbulent flow around MIRA model using linear equal‐order finite element method

A parallel large eddy simulation code that adopts domain decomposition method has been developed for large‐scale computation of turbulent flows around an arbitrarily shaped body. For the temporal integration of the unsteady incompressible Navier–Stokes equation, fractional 4‐step splitting algorithm is adopted, and for the modelling of small eddies in turbulent flows, the Smagorinsky model is used. For the parallelization of the code, METIS and Message Passing Interface Libraries are used, respectively, to partition the computational domain and to communicate data between processors. To validate the parallel architecture and to estimate its performance, a three‐dimensional laminar driven cavity flow inside a cubical enclosure has been solved. To validate the turbulence calculation, the turbulent channel flows at Re_τ = 180 and 1050 are simulated and compared with previous results. Then, a backward facing step flow is solved and compared with a DNS result for overall code validation. Finally, the turbulent flow around MIRA model at Re = 2.6 × 10⁶ is simulated by using approximately 6.7 million nodes. Scalability curve obtained from this simulation shows that scalable results are obtained. The calculated drag coefficient agrees better with the experimental result than those previously obtained by using two‐equation turbulence models. Copyright © 2007 John Wiley & Sons, Ltd.     

A method for finite element parallel viscous compressible flow calculations

This paper presents the parallelization aspects of a solution method for the fully coupled 3D compressible Navier-Stokes equations. The algorithmic thrust of the approach, embedded in a finite element code NS3D, is the linearization of the governing equations through Newton methods, followed by a fully coupled solution of velocities and pressure at each non-linear iteration by preconditioned conjugate gradient-like iterative algorithms. For the matrix assembly, as well as for the linear equation solver, efficient coarse-grain parallel schemes have been developed for shared memory machines, as well as for networks of workstations, with a moderate number of processors. The parallel iterative schemes, in particular, circumvent some of the difficulties associated with domain decomposition methods, such as geometry bookkeeping and the sometimes drastic convergence slow-down of partitioned non-linear problems.