Performance of a parallel algebraic multilevel preconditioner for stabilized finite element semiconductor device modeling

In this study results are presented for the large-scale parallel performance of an algebraic multilevel preconditioner for solution of the drift-diffusion model for semiconductor devices. The preconditioner is the key numerical procedure determining the robustness, efficiency and scalability of the fully-coupled Newton–Krylov based, nonlinear solution method that is employed for this system of equations. The coupled system is comprised of a source term dominated Poisson equation for the electric potential, and two convection–diffusion-reaction type equations for the electron and hole concentration. The governing PDEs are discretized in space by a stabilized finite element method. Solution of the discrete system is obtained through a fully-implicit time integrator, a fully-coupled Newton-based nonlinear solver, and a restarted GMRES Krylov linear system solver. The algebraic multilevel preconditioner is based on an aggressive coarsening graph partitioning of the nonzero block structure of the Jacobian matrix. Representative performance results are presented for various choices of multigrid V-cycles and W-cycles and parameter variations for smoothers based on incomplete factorizations. Parallel scalability results are presented for solution of up to 10⁸${10}^8$ unknowns on 4096 processors of a Cray XT3/4 and an IBM POWER eServer system.     

Towards large-scale multi-socket,multicore parallel simulations: Performance of an MPI-only semiconductor device simulator

This preliminary study considers the scaling and performance of a finite element (FE) semiconductor device simulator on a set of multi-socket, multicore architectures with nonuniform memory access (NUMA) compute nodes. These multicore architectures include two linux clusters with multicore processors: a quad-socket, quad-core AMD Opteron platform and a dual-socket, quad-core Intel Xeon Nehalem platform; and a dual-socket, six-core AMD Opteron workstation. These platforms have complex memory hierarchies that include local core-based cache, local socket-based memory, access to memory on the same mainboard from another socket, and then memory across network links to different nodes. The specific semiconductor device simulator used in this study employs a fully-coupled Newton–Krylov solver with domain decomposition and multilevel preconditioners. Scaling results presented include a large-scale problem of 100+ million unknowns on 4096 cores and a comparison with the Cray XT3/4 Red Storm capability platform. Although the MPI-only device simulator employed for this work can take advantage of all the cores of quad-core and six-core CPUs, the efficiency of the linear system solve is decreasing with increased core count and eventually a different programming paradigm will be needed.     

Sub-iteration leads to accuracy and stability enhancements of semi-implicit schemes for the Navier–Stokes equations

We present an iterative semi-implicit scheme for the incompressible Navier–Stokes equations, which is stable at CFL numbers well above the nominal limit. We have implemented this scheme in conjunction with spectral discretizations, which suffer from serious time step limitations at very high resolution. However, the approach we present is general and can be adopted with finite element and finite difference discretizations as well. Specifically, at each time level, the nonlinear convective term and the pressure boundary condition – both of which are treated explicitly in time – are updated using fixed-point iteration and Aitken relaxation. Eigenvalue analysis shows that this scheme is unconditionally stable for Stokes flows while numerical results suggest that the same is true for steady Navier–Stokes flows as well. This finding is also supported by error analysis that leads to the proper value of the relaxation parameter as a function of the flow parameters. In unsteady flows, second- and third-order temporal accuracy is obtained for the velocity field at CFL number 5–14 using analytical solutions. Systematic accuracy, stability, and cost comparisons are presented against the standard semi-implicit method and a recently proposed fully-implicit scheme that does not require Newton’s iterations. In addition to its enhanced accuracy and stability, the proposed method requires the solution of symmetric only linear systems for which very effective preconditioners exist unlike the fully-implicit schemes.     

Approximation of the inductionless MHD problem using a stabilized finite element method

In this work, we present a stabilized formulation to solve the inductionless magnetohydrodynamic (MHD) problem using the finite element (FE) method. The MHD problem couples the Navier–Stokes equations and a Darcy-type system for the electric potential via Lorentz’s force in the momentum equation of the Navier–Stokes equations and the currents generated by the moving fluid in Ohm’s law. The key feature of the FE formulation resides in the design of the stabilization terms, which serve several purposes. First, the formulation is suitable for convection dominated flows. Second, there is no need to use interpolation spaces constrained to a compatibility condition in both sub-problems and therefore, equal-order interpolation spaces can be used for all the unknowns. Finally, this formulation leads to a coupled linear system; this monolithic approach is effective, since the coupling can be dealt by effective preconditioning and iterative solvers that allows to deal with high Hartmann numbers.     

Nonlinear finite element model updating of an infilled frame based on identified time-varying modal parameters during an earthquake

A model updating methodology is proposed for calibration of nonlinear finite element (FE) models simulating the behavior of real-world complex civil structures subjected to seismic excitations. In the proposed methodology, parameters of hysteretic material models assigned to elements (or substructures) of a nonlinear FE model are updated by minimizing an objective function. The objective function used in this study is the misfit between the experimentally identified time-varying modal parameters of the structure and those of the FE model at selected time instances along the response time history. The time-varying modal parameters are estimated using the deterministic–stochastic subspace identification method which is an input–output system identification approach. The performance of the proposed updating method is evaluated through numerical and experimental applications on a large-scale three-story reinforced concrete frame with masonry infills. The test structure was subjected to seismic base excitations of increasing amplitude at a large outdoor shake-table. A nonlinear FE model of the test structure has been calibrated to match the time-varying modal parameters of the test structure identified from measured data during a seismic base excitation. The accuracy of the proposed nonlinear FE model updating procedure is quantified in numerical and experimental applications using different error metrics. The calibrated models predict the exact simulated response very accurately in the numerical application, while the updated models match the measured response reasonably well in the experimental application.     

XTOR-2F: A fully implicit Newton–Krylov solver applied to nonlinear 3D extended MHD in tokamaks

XTOR-2F solves a set of extended magnetohydrodynamic (MHD) equations in toroidal tokamak geometry. In the original XTOR code, the time stepping is handled by a semi-implicit method 1, 2 and 3. Moderate changes were necessary to transform it into a fully implicit one using the NITSOL library with Newton–Krylov methods of solution for nonlinear system of equations [4]. After addressing the sensitive issue of preconditioning and time step tuning, the performances of the semi-implicit and the implicit methods are compared for the nonlinear simulation of an internal kink mode test case within the framework of resistive MHD including anisotropic thermal transport. A convergence study comparing the semi-implicit and the implicit schemes is presented. Our main conclusion is that on one hand the Newton–Krylov implicit method, when applied to basic one fluid MHD is more computationally costly than the semi-implicit one by a factor 3 for a given numerical accuracy. But on the other hand, the implicit method allows to address challenging issues beyond MHD. By testing the Newton–Krylov method with diamagnetic modifications on the dynamics of the internal kink, some numerical issues, to be addressed further, are emphasized.     

Algebraic Multigrid Preconditioning for Finite Element Solution of Inhomogeneous Elastic Inclusion Problems in Articular Cartilage

In studying biomechanical deformation in articular cartilage, the presence of cells (chondrocytes) necessitates the consideration of inhomogeneous elasticity problems in which cells are idealized as soft inclusions within a stiff extracellular matrix. An analytical solution of a soft inclusion problem is derived and used to evaluate iterative numerical solutions of the associated linear algebraic system based on discretization via the finite element method, and use of an iterative conjugate gradient method with algebraic multigrid preconditioning (AMG-PCG). Accuracy and efficiency of the AMG-PCG algorithm is compared to two other conjugate gradient algorithms with diagonal preconditioning (DS-PCG) or a modified incomplete LU decomposition (Euclid-PCG) based on comparison to the analytical solution. While all three algorithms are shown to be accurate, the AMG-PCG algorithm is demonstrated to provide significant savings in CPU time as the number of nodal unknowns is increased. In contrast to the other two algorithms, the AMG-PCG algorithm also exhibits little sensitivity of CPU time and number of iterations to variations in material properties that are known to significantly affect model variables. Results demonstrate the benefits of algebraic multigrid preconditioners for the iterative solution of assembled linear systems based on finite element modeling of soft elastic inclusion problems and may be particularly advantageous for large scale problems with many nodal unknowns.     

Wavelet based ILU preconditioners for the numerical solution by PUFEM of high frequency elastic wave scattering

Daubechies family of wavelets combined to the Incomplete Lower–Upper (ILU) factorization are considered as preconditioners for a block sparse linear system arising from the approximation of the time harmonic elastic wave equations by the Partition of Unity Finite Element Method (PUFEM). After applying the discrete wavelet transform (DWT) to each dense block in the final matrix and the known right-hand side, due to the local enrichment by pressure (P) and shear (S) plane waves, the resulting linear system is solved by the restarted Generalized Minimum RESidual method (GMRES) with ILU preconditioners allowing fill-in elements in the L and U matrix factors. A reordering algorithm of the vertices based on Gibbs method is also introduced. It leads to a significant reduction of the bandwidth of the wave based Finite Element (FE) matrix and enables the ILU preconditioners to be more effective. To study the performance of the proposed preconditioners, a problem of a horizontal S plane wave scattered by a rigid circular body in an infinite elastic medium is considered. The numerical tests show the good performance of the DWT based ILU preconditioners in improving the rate of convergence of GMRES, for high numbers of approximating P and S plane waves, on coarse mesh grids containing multi-wavelength sized elements. Moreover, the Haar DWT enhances the scalability with respect to the problem size, when the number of the nodal points increases, of the ILU preconditioner which uses the threshold strategy in the control of the fill-in elements. Despite the high level of the conditioning, a good accuracy may be achieved for a discretization level of around 1.9 degrees of freedom per S wavelength, which is far below the rule of thumb of 10 nodal points per wavelength, adopted for the conventional FE.     

A cell functional minimization scheme for parabolic problem

We are interested in a robust and accurate finite volume scheme for 2-D parabolic problems derived from the cell functional minimization approach. The scheme has a local stencil, is locally conservative, treats discontinuity rigorously and leads to a symmetric positive definite linear system. Since the scheme has both cell centered unknowns and cell edge unknowns, the computational cost is an issue and a parallel algorithm is then suggested based on nonoverlapping domain decomposition approach. The interface condition is of the Dirichlet–Robin type and has a parameter λ. By choosing this parameter properly, the convergence of the iteration process could be sped up. Numerical results for linear and nonlinear problems demonstrate the good performance of the cell functional minimization scheme and its parallel version on distorted meshes.     

Stabilization and scalable block preconditioning for the Navier–Stokes equations

This study compares several block-oriented preconditioners for the stabilized finite element discretization of the incompressible Navier–Stokes equations. This includes standard additive Schwarz domain decomposition methods, aggressive coarsening multigrid, and three preconditioners based on an approximate block LU factorization, specifically SIMPLEC, LSC, and PCD. Robustness is considered with a particular focus on the impact that different stabilization methods have on preconditioner performance. Additionally, parallel scaling studies are undertaken. The numerical results indicate that aggressive coarsening multigrid, LSC and PCD all have good algorithmic scalability. Coupling this with the fact that block methods can be applied to systems arising from stable mixed discretizations implies that these techniques are a promising direction for developing scalable methods for Navier–Stokes.     

Toward textbook multigrid efficiency for fully implicit resistive magnetohydrodynamics

Multigrid methods can solve some classes of elliptic and parabolic equations to accuracy below the truncation error with a work-cost equivalent to a few residual calculations – so-called “textbook” multigrid efficiency. We investigate methods to solve the system of equations that arise in time dependent magnetohydrodynamics (MHD) simulations with textbook multigrid efficiency. We apply multigrid techniques such as geometric interpolation, full approximate storage, Gauss–Seidel smoothers, and defect correction for fully implicit, nonlinear, second-order finite volume discretizations of MHD. We apply these methods to a standard resistive MHD benchmark problem, the GEM reconnection problem, and add a strong magnetic guide field, which is a critical characteristic of magnetically confined fusion plasmas. We show that our multigrid methods can achieve near textbook efficiency on fully implicit resistive MHD simulations.     

An adaptive MHD method for global space weather simulations

A 3D parallel adaptive mesh refinement (AMR) scheme is described for solving the partial-differential equations governing ideal magnetohydrodynamic (MHD) flows. This new algorithm adopts a cell-centered upwind finite-volume discretization procedure and uses limited solution reconstruction, approximate Riemann solvers, and explicit multi-stage time stepping to solve the MHD equations in divergence form, providing a combination of high solution accuracy and computational robustness across a large range in the plasma β (β is the ratio of thermal and magnetic pressures). The data structure naturally lends itself to domain decomposition, thereby enabling efficient and scalable implementations on massively parallel supercomputers. Numerical results for MHD simulations of magnetospheric plasma flows are described to demonstrate the validity and capabilities of the approach for space weather applications     

Completely two-dimensional model for examining the characteristics of a linear cylindrical induction pump

A completely two-dimensional mathematical model for calculating the characteristics of induction magnetichydrodynamic (MHD) machines with a cylindrical channel is proposed. The flow pattern of a liquid metal in an electromagnetic pump under MHD instability is obtained by numeric analysis. This pattern is characterized by the formation of large-scale vortices traveling longitudinally and azimuthally. The calculated basic characteristics of the pump are in good qualitative and in satisfactory quantitative agreement with the experiment.     

Preconditioning methods for discontinuous Galerkin solutions of the Navier–Stokes equations

A Newton–Krylov method is developed for the solution of the steady compressible Navier–Stokes equations using a discontinuous Galerkin (DG) discretization on unstructured meshes. Steady-state solutions are obtained using a Newton–Krylov approach where the linear system at each iteration is solved using a restarted GMRES algorithm. Several different preconditioners are examined to achieve fast convergence of the GMRES algorithm. An element Line-Jacobi preconditioner is presented which solves a block-tridiagonal system along lines of maximum coupling in the flow. An incomplete block-LU factorization (Block-ILU(0)) is also presented as a preconditioner, where the factorization is performed using a reordering of elements based upon the lines of maximum coupling. This reordering is shown to be superior to standard reordering techniques (Nested Dissection, One-way Dissection, Quotient Minimum Degree, Reverse Cuthill–Mckee) especially for viscous test cases. The Block-ILU(0) factorization is performed in-place and an algorithm is presented for the application of the linearization which reduces both the memory and CPU time over the traditional dual matrix storage format. Additionally, a linear p-multigrid preconditioner is also considered, where Block-Jacobi, Line-Jacobi and Block-ILU(0) are used as smoothers. The linear multigrid preconditioner is shown to significantly improve convergence in term of number of iterations and CPU time compared to a single-level Block-Jacobi or Line-Jacobi preconditioner. Similarly the linear multigrid preconditioner with Block-ILU smoothing is shown to reduce the number of linear iterations to achieve convergence over a single-level Block-ILU(0) preconditioner, though no appreciable improvement in CPU time is shown.     

Faraday laser at Rb 1529 nm transition for optical communication systems

We experimentally demonstrate a Faraday laser at Rb 1529 nm transition by using a performance-improved Rb electrodeless-discharge-lamp-based excited-state Faraday anomalous dispersion optical filter as the frequencyselective element. Neither the electrical locking scheme nor the additional frequency-stabilized pump laser are used. The frequency of the external-cavity diode laser is stabilized to the Rb 1529 nm transition, and the Allan deviation of the Faraday laser is measured by converting the optical intensity into frequency. The Faraday laser can be used as a frequency standard in the telecom C band for further research on metrology, microwave photonics, and optical communication systems.     

Peculiarities of the inverse Faraday effect induced in iron garnet films by femtosecond laser pulses

The inverse Faraday effect in iron garnet films subjected to femtosecond laser pulses is experimentally investigated. It is found that the magnitude of the observed effect depends nonlinearly on the energy of the optical pump pulses, which is in contradiction with the notion that the inverse Faraday effect is linear with respect to the pump energy. Thus, for pump pulses with a central wavelength of 650 nm and an energy density of 1 mJ/cm², the deviation from a linear dependence is as large as 50%. Analysis of the experimental data demonstrates that the observed behavior is explained by the fact that the optically induced normal component of the magnetization is determined, apart from the field resulting from the inverse Faraday effect, by a decrease in the magnitude of the precessing magnetization under the influence of the femtosecond electromagnetic field.     

Three dimensional dynamics of pretwisted beams: A spectral-Tchebychev solution

This paper presents the application of the spectral-Tchebychev (ST) technique for solution of three-dimensional dynamics of unconstrained pretwisted beams with general cross-section (including both straight and curved cross-sections). In general, the dynamic response of pretwisted beams presents three-dimensional (3D) motions, including coupled bending–bending–torsional–axial motions. As such, accurately solving pretwisted beam dynamics requires a 3D solution approach. In this work, the integral boundary value problem based on the 3D linear elasticity equations is solved numerically using the 3D-ST approach. To simplify evaluation of the volume integrals, the boundaries are simplified by applying two coordinate transformations to render the pretwisted beam with curved cross-section into an equivalent straight beam with rectangular cross-section. Three sample pretwisted beam problems with rectangular, curved, and airfoil cross-sections at different twist rates are solved using the presented approach. In each case, the convergence of the solution is analyzed, and non-dimensional natural frequencies and mode shapes are compared to those from a finite-element (FE) solution. Furthermore, cross-sectional stress and displacements are obtained from the 3D-ST solution. Lastly, the non-dimensional natural frequencies from the 3D-ST and a 1D/2D solutions are compared. It is concluded that the 3D-ST solution can capture the three-dimensional dynamic behavior of pretwisted beams as accurately as an FE solution, but for a fraction of the computational cost. Furthermore, it is shown that 1D/2D solution can lead to significant errors at high twist rates, and thus, the 3D-ST solution should be preferred.     

Parallel multilevel methods for implicit solution of shallow water equations with nonsmooth topography on the cubed-sphere

High resolution and scalable parallel algorithms for the shallow water equations on the sphere are very important for modeling the global climate. In this paper, we introduce and study some highly scalable multilevel domain decomposition methods for the fully implicit solution of the nonlinear shallow water equations discretized with a second-order well-balanced finite volume method on the cubed-sphere. With the fully implicit approach, the time step size is no longer limited by the stability condition, and with the multilevel preconditioners, good scalabilities are obtained on computers with a large number of processors. The investigation focuses on the use of semismooth inexact Newton method for the case with nonsmooth topography and the use of two- and three-level overlapping Schwarz methods with different order of discretizations for the preconditioning of the Jacobian systems. We test the proposed algorithm for several benchmark cases and show numerically that this approach converges well with smooth and nonsmooth bottom topography, and scales perfectly in terms of the strong scalability and reasonably well in terms of the weak scalability on machines with thousands and tens of thousands of processors.     

Fast volumetric integral-equation solver for acoustic wave propagation through inhomogeneous media

Elements are described of a volumetric integral-equation-based algorithm applicable to accurate large-scale simulations of scattering and propagation of sound waves through inhomogeneous media. The considered algorithm makes possible simulations involving realistic geometries characterized by highly subwavelength details, large density contrasts, and described in terms of several million unknowns. The algorithm achieves its competitive performance, characterized by O(N log N) solution complexity and O(N) memory requirements, where N is the number of unknowns, through a fast and nonlossy fast Fourier transform based matrix compression technique, the adaptive integral method, previously developed for solving large-scale electromagnetic problems. Because of its ability of handling large problems with complex geometries, the developed solver may constitute an efficient and high fidelity numerical simulation tool for calculating acoustic field distributions in anatomically realistic models, e.g., in investigating acoustic energy transfer to the inner ear via nonairborne pathways in the human head. Examples of calculations of acoustic field distribution in a human head, which require solving linear systems of equations involving several million unknowns, are presented.