Midpoint cell method for hybrid (MPI+OpenMP) parallelization of molecular dynamics simulations

We have developed a new hybrid (MPI+OpenMP) parallelization scheme for molecular dynamics (MD) simulations by combining a cell‐wise version of the midpoint method with pair‐wise Verlet lists. In this scheme, which we call the midpoint cell method, simulation space is divided into subdomains, each of which is assigned to a MPI processor. Each subdomain is further divided into small cells. The interaction between two particles existing in different cells is computed in the subdomain containing the midpoint cell of the two cells where the particles reside. In each MPI processor, cell pairs are distributed over OpenMP threads for shared memory parallelization. The midpoint cell method keeps the advantages of the original midpoint method, while filtering out unnecessary calculations of midpoint checking for all the particle pairs by single midpoint cell determination prior to MD simulations. Distributing cell pairs over OpenMP threads allows for more efficient shared memory parallelization compared with distributing atom indices over threads. Furthermore, cell grouping of particle data makes better memory access, reducing the number of cache misses. The parallel performance of the midpoint cell method on the K computer showed scalability up to 512 and 32,768 cores for systems of 20,000 and 1 million atoms, respectively. One MD time step for long‐range interactions could be calculated within 4.5 ms even for a 1 million atoms system with particle‐mesh Ewald electrostatics. © 2014 Wiley Periodicals, Inc.     

Hybrid particle‐field molecular dynamics simulations: Parallelization and benchmarks

The parallel implementation of a recently developed hybrid scheme for molecular dynamics (MD) simulations (Milano and Kawakatsu, J Chem Phys 2009, 130, 214106) where self‐consistent field theory (SCF) and particle models are combined is described. Because of the peculiar formulation of the hybrid method, considering single particles interacting with density fields, the most computationally expensive part of the hybrid particle‐field MD simulation can be efficiently parallelized using a straightforward particle decomposition algorithm. Benchmarks of simulations, including comparisons of serial MD and MD‐SCF program profiles, serial MD‐SCF and parallel MD‐SCF program profiles, and parallel benchmarks compared with efficient MD program GROMACS 4.5.4 are tested and reported. The results of benchmarks indicate that the proposed parallelization scheme is very efficient and opens the way to molecular simulations of large scale systems with reasonable computational costs. © 2012 Wiley Periodicals, Inc.     

GENESIS 1.1: A hybrid‐parallel molecular dynamics simulator with enhanced sampling algorithms on multiple computational platforms

GENeralized‐Ensemble SImulation System (GENESIS) is a software package for molecular dynamics (MD) simulation of biological systems. It is designed to extend limitations in system size and accessible time scale by adopting highly parallelized schemes and enhanced conformational sampling algorithms. In this new version, GENESIS 1.1, new functions and advanced algorithms have been added. The all‐atom and coarse‐grained potential energy functions used in AMBER and GROMACS packages now become available in addition to CHARMM energy functions. The performance of MD simulations has been greatly improved by further optimization, multiple time‐step integration, and hybrid (CPU + GPU) computing. The string method and replica‐exchange umbrella sampling with flexible collective variable choice are used for finding the minimum free‐energy pathway and obtaining free‐energy profiles for conformational changes of a macromolecule. These new features increase the usefulness and power of GENESIS for modeling and simulation in biological research. © 2017 Wiley Periodicals, Inc.     

Structural, dynamic, and electrostatic properties of fully hydrated DMPC bilayers from molecular dynamics simulations accelerated with graphical processing units (GPUs)

We present results of molecular dynamics simulations of fully hydrated DMPC bilayers performed on graphics processing units (GPUs) using current state-of-the-art non-polarizable force fields and a local GPU-enabled molecular dynamics code named FEN ZI. We treat the conditionally convergent electrostatic interaction energy exactly using the particle mesh Ewald method (PME) for solution of Poisson's Equation for the electrostatic potential under periodic boundary conditions. We discuss elements of our implementation of the PME algorithm on GPUs as well as pertinent performance issues. We proceed to show results of simulations of extended lipid bilayer systems using our program, FEN ZI. We performed simulations of DMPC bilayer systems consisting of 17,004, 68,484, and 273,936 atoms in explicit solvent. We present bilayer structural properties (atomic number densities, electron density profiles), deuterium order parameters (S(CD)), electrostatic properties (dipole potential, water dipole moments), and orientational properties of water. Predicted properties demonstrate excellent agreement with experiment and previous all-atom molecular dynamics simulations. We observe no statistically significant differences in calculated structural or electrostatic properties for different system sizes, suggesting the small bilayer simulations (less than 100 lipid molecules) provide equivalent representation of structural and electrostatic properties associated with significantly larger systems (over 1000 lipid molecules). We stress that the three system size representations will have differences in other properties such as surface capillary wave dynamics or surface tension related effects that are not probed in the current study. The latter properties are inherently dependent on system size. This contribution suggests the suitability of applying emerging GPU technologies to studies of an important class of biological environments, that of lipid bilayers and their associated integral membrane proteins. We envision that this technology will push the boundaries of fully atomic-resolution modeling of these biological systems, thus enabling unprecedented exploration of meso-scale phenomena (mechanisms, kinetics, energetics) with atomic detail at commodity hardware prices.     

Hierarchical parallelization of divide-and-conquer density functional tight-binding molecular dynamics and metadynamics simulations

Massively parallel divide-and-conquer density functional tight-binding (DC-DFTB) molecular dynamics and metadynamics simulations are efficient approaches for describing various chemical reactions and dynamic processes of large complex systems via quantum mechanics. In this study, DC-DFTB simulations were combined with multi-replica techniques. Specifically, multiple walkers metadynamics, replica exchange molecular dynamics, and parallel tempering metadynamics methods were implemented hierarchically into the in-house Dcdftbmd program. Test simulations in an aqueous phase of the internal rotation of formamide and conformational changes of dialanine showed that the newly developed extensions increase the sampling efficiency and the exploration capabilities in DC-DFTB configuration space.     

The distributed diagonal force decomposition method for parallelizing molecular dynamics simulations

Parallelization is an effective way to reduce the computational time needed for molecular dynamics simulations. We describe a new parallelization method, the distributed-diagonal force decomposition method, with which we extend and improve the existing force decomposition methods. Our new method requires less data communication during molecular dynamics simulations than replicated data and current force decomposition methods, increasing the parallel efficiency. It also dynamically load-balances the processors' computational load throughout the simulation. The method is readily implemented in existing molecular dynamics codes and it has been incorporated into the CHARMM program, allowing its immediate use in conjunction with the many molecular dynamics simulation techniques that are already present in the program. We also present the design of the Force Decomposition Machine, a cluster of personal computers and networks that is tailored to running molecular dynamics simulations using the distributed diagonal force decomposition method. The design is expandable and provides various degrees of fault resilience. This approach is easily adaptable to computers with Graphics Processing Units because it is independent of the processor type being used.     

Coarse‐grained molecular dynamics simulations of polymerization with forward and backward reactions

We develop novel parallel algorithms that allow molecular dynamics simulations in which byproduct molecules are created and removed because of the chemical reactions during the molecular dynamics simulation. To prevent large increases in the potential energy, we introduce the byproduct molecules smoothly by changing the non‐bonded interactions gradually. To simulate complete equilibrium reactions, we allow the byproduct molecules attack and destroy created bonds. Modeling of such reactions are, for instance, important to study the pore formation due to the presence of e.g. water molecules or development of polymer morphology during the process of splitting off byproduct molecules. Another concept that could be studied is the degradation of polymeric materials, a very important topic in a recycling of polymer waste. We illustrate the method by simulating the polymerization of polyethylene terephthalate (PET) at the coarse‐grained level as an example of a polycondensation reaction with water as a byproduct. The algorithms are implemented in a publicly available software package and are easily accessible using a domain‐specific language that describes chemical reactions in an input configuration file. © 2018 Wiley Periodicals, Inc.     

Efficient nonbonded interactions for molecular dynamics on a graphics processing unit

We describe an algorithm for computing nonbonded interactions with cutoffs on a graphics processing unit. We have incorporated it into OpenMM, a library for performing molecular simulations on high‐performance computer architectures. We benchmark it on a variety of systems including boxes of water molecules, proteins in explicit solvent, a lipid bilayer, and proteins with implicit solvent. The results demonstrate that its performance scales linearly with the number of atoms over a wide range of system sizes, while being significantly faster than other published algorithms. © 2009 Wiley Periodicals, Inc. J Comput Chem, 2010     

Algorithms for GPU‐based molecular dynamics simulations of complex fluids: Applications to water,mixtures, and liquid crystals

A custom code for molecular dynamics simulations has been designed to run on CUDA‐enabled NVIDIA graphics processing units (GPUs). The double‐precision code simulates multicomponent fluids, with intramolecular and intermolecular forces, coarse‐grained and atomistic models, holonomic constraints, Nosé–Hoover thermostats, and the generation of distribution functions. Algorithms to compute Lennard‐Jones and Gay‐Berne interactions, and the electrostatic force using Ewald summations, are discussed. A neighbor list is introduced to improve scaling with respect to system size. Three test systems are examined: SPC/E water; an n‐hexane/2‐propanol mixture; and a liquid crystal mesogen, 2‐(4‐butyloxyphenyl)‐5‐octyloxypyrimidine. Code performance is analyzed for each system. With one GPU, a 33–119 fold increase in performance is achieved compared with the serial code while the use of two GPUs leads to a 69–287 fold improvement and three GPUs yield a 101–377 fold speedup. © 2015 Wiley Periodicals, Inc.     

Scalable fine-grained parallelization of plane-wave-based ab initio molecular dynamics for large supercomputers

Many systems of great importance in material science, chemistry, solid-state physics, and biophysics require forces generated from an electronic structure calculation, as opposed to an empirically derived force law to describe their properties adequately. The use of such forces as input to Newton's equations of motion forms the basis of the ab initio molecular dynamics method, which is able to treat the dynamics of chemical bond-breaking and -forming events. However, a very large number of electronic structure calculations must be performed to compute an ab initio molecular dynamics trajectory, making the efficiency as well as the accuracy of the electronic structure representation critical issues. One efficient and accurate electronic structure method is the generalized gradient approximation to the Kohn-Sham density functional theory implemented using a plane-wave basis set and atomic pseudopotentials. The marriage of the gradient-corrected density functional approach with molecular dynamics, as pioneered by Car and Parrinello (R. Car and M. Parrinello, Phys Rev Lett 1985, 55, 2471), has been demonstrated to be capable of elucidating the atomic scale structure and dynamics underlying many complex systems at finite temperature. However, despite the relative efficiency of this approach, it has not been possible to obtain parallel scaling of the technique beyond several hundred processors on moderately sized systems using standard approaches. Consequently, the time scales that can be accessed and the degree of phase space sampling are severely limited. To take advantage of next generation computer platforms with thousands of processors such as IBM's BlueGene, a novel scalable parallelization strategy for Car-Parrinello molecular dynamics is developed using the concept of processor virtualization as embodied by the Charm++ parallel programming system. Charm++ allows the diverse elements of a Car-Parrinello molecular dynamics calculation to be interleaved with low latency such that unprecedented scaling is achieved. As a benchmark, a system of 32 water molecules, a common system size employed in the study of the aqueous solvation and chemistry of small molecules, is shown to scale on more than 1500 processors, which is impossible to achieve using standard approaches. This degree of parallel scaling is expected to open new opportunities for scientific inquiry.     

Molecular dynamics simulations using the drude polarizable force field on GPUs with OpenMM: Implementation,validation, and benchmarks

Presented is the implementation of the Drude force field in the open‐source OpenMM simulation package allowing for access to graphical processing unit (GPU) hardware. In the Drude model, electronic degrees of freedom are represented by negatively charged particles attached to their parent atoms via harmonic springs, such that extra computational overhead comes from these additional particles and virtual sites representing lone pairs on electronegative atoms, as well as the associated thermostat and integration algorithms. This leads to an approximately fourfold increase in computational demand over additive force fields. However, by making the Drude model accessible to consumer‐grade desktop GPU hardware it will be possible to perform simulations of one microsecond or more in less than a month, indicating that the barrier to employ polarizable models has largely been removed such that polarizable simulations with the classical Drude model are readily accessible and practical.     

A box-counting-based algorithm for computing Shannon entropy in molecular dynamics simulations

A box-counting-based algorithm (SEBC) has been developed for the numerical computation of the Shannon entropy from samples of continuous functions. Its performance was tested by applying it to several samples of known continuous distribution functions. The results obtained with SEBC reproduced those obtained by analytical or numerical integration. SEBC was also employed for computing the Shannon entropies of the steric energy, Sh(E(S)), of several amino acids from their in vacuo NVE molecular dynamics simulations using the AMBER-4 force field. The results obtained correlate linearly with the experimental standard thermodynamic entropies of these compounds. This work points to the possibility of introducing straightforward and reliable calculations of thermodynamic entropies from empirical linear relationships with Sh(E(S)) obtained from MD simulations.     

A comparison of different initialization protocols to obtain statistically independent molecular dynamics simulations

We study how the results of molecular dynamics (MD) simulations are affected by various choices during the setup, e.g., the starting velocities, the solvation, the location of protons, the conformation of His, Asn, and Gln residues, the protonation and titration of His residues, and the treatment of alternative conformations. We estimate the binding affinity of ligands to four proteins calculated with the MM/GBSA method (molecular mechanics combined with a generalized Born and surface area solvation energy). For avidin and T4 lysozyme, all variations gave similar results within 2 kJ/mol. For factor Xa, differences in the solvation or in the selection of alternative conformations gave results that are significantly different from those of the other approaches by 4-6 kJ/mol, whereas for galectin-3, changes in the conformations, rotations, and protonation gave results that differed by 10 kJ/mol, but only if residues close to the binding site were modified. This shows that the results of MM/GBSA calculations are reasonably reproducible even if the MD simulations are set up with different software. Moreover, we show that the sampling of phase space can be enhanced by solvating the systems with different equilibrated water boxes, in addition to the common use of different starting velocities. If different conformations are available in the crystal structure, they should also be employed to enhance the sampling. Protonation, ionization, and conformations of Asn, Gln, and His may also be used to enhance sampling, but great effort should be spent to obtain as reliable predictions as possible close to the active site.     

Investigation of chiral recognition by molecular micelles with molecular dynamics simulations

Molecular dynamics simulations were used to characterize the binding of the chiral drugs chlorthalidone and lorazepam to the molecular micelle poly-(sodium undecyl-(L)-leucine-valine). The project’s goal was to characterize the nature of chiral recognition in capillary electrophoresis separations that use molecular micelles as the chiral selector. The shapes and charge distributions of the chiral molecules investigated, their orientations within the molecular micelle chiral binding pockets, and the formation of stereoselective intermolecular hydrogen bonds with the molecular micelle were all found to play key roles in determining where and how lorazepam and chlorthalidone enantiomers interacted with the molecular micelle.     

Explicit proton transfer in classical molecular dynamics simulations

We present Hydrogen Dynamics (HYDYN), a method that allows explicit proton transfer in classical force field molecular dynamics simulations at thermodynamic equilibrium. HYDYN reproduces the characteristic properties of the excess proton in water, from the special pair dance, to the continuous fluctuation between the limiting Eigen and Zundel complexes, and the water reorientation beyond the first solvation layer. Advantages of HYDYN with respect to existing methods are computational efficiency, microscopic reversibility, and easy parameterization for any force field. © 2014 Wiley Periodicals, Inc.     

Steered molecular dynamics simulations of protein-ligand interactions

Molecular recognition and specific protein-ligandinteractions are central to many biochemical processes,such as enzyme catalysis, assembly of organelles, en-ergy transduction, signaling, diverse control functions,and replication, expression and storage of the geneticmaterial[1]. Moreover, protein-ligand interactions pro-vide the mechanism of many drug therapies and un-derstanding of such interactions is thus significant forrational drug design[1,2]. For the experimental studiesof protein-ligan…     

A new set of molecular mechanics parameters for hydroxyproline and its use in molecular dynamics simulations of collagen-like peptides

Recently, the importance of proline ring pucker conformations in collagen has been suggested in the context of hydroxylation of prolines. The previous molecular mechanics parameters for hydroxyproline, however, do not reproduce the correct pucker preference. We have developed a new set of parameters that reproduces the correct pucker preference. Our molecular dynamics simulations of proline and hydroxyproline monomers as well as collagen-like peptides, using the new parameters, support the theory that the role of hydroxylation in collagen is to stabilize the triple helix by adjusting to the right pucker conformation (and thus the right phi angle) in the Y position.     

Looking inside the entanglement “tube” using molecular dynamics simulations

For 30 years, the dynamics of entangled polymers have been explained using the phenomenological “tube” model, where the “tube” represents the confining effects of surrounding chains, but the tube properties, such as its length and diameter, could only be inferred indirectly by fitting the tube model to rheological data. Now, however, molecular simulations are allowing these properties to be directly computed. The computational advances in molecular dynamics and related methods that have made this possible are here reviewed. In addition, it is discussed how new findings, such as an apparent time dependence of the tube diameter and direct observation of “hopping” of branch points along the tube, are helping to refine the tube model. © 2007 Wiley Periodicals, Inc. J Polym Sci Part B: Polym Phys 45: 3240–3248, 2007     

Accelerated molecular dynamics simulations of protein folding

Folding of four fast‐folding proteins, including chignolin, Trp‐cage, villin headpiece and WW domain, was simulated via accelerated molecular dynamics (aMD). In comparison with hundred‐of‐microsecond timescale conventional molecular dynamics (cMD) simulations performed on the Anton supercomputer, aMD captured complete folding of the four proteins in significantly shorter simulation time. The folded protein conformations were found within 0.2–2.1 Å of the native NMR or X‐ray crystal structures. Free energy profiles calculated through improved reweighting of the aMD simulations using cumulant expansion to the second‐order are in good agreement with those obtained from cMD simulations. This allows us to identify distinct conformational states (e.g., unfolded and intermediate) other than the native structure and the protein folding energy barriers. Detailed analysis of protein secondary structures and local key residue interactions provided important insights into the protein folding pathways. Furthermore, the selections of force fields and aMD simulation parameters are discussed in detail. Our work shows usefulness and accuracy of aMD in studying protein folding, providing basic references in using aMD in future protein‐folding studies. © 2015 Wiley Periodicals, Inc.     

A simple algorithm to accelerate the computation of non-bonded interactions in cell-based molecular dynamics simulations

Cell lists are ubiquitous in molecular dynamics simulations--be it for the direct computation of short-range inter-atomic potentials, the short-range direct part of a long-range interaction or for the periodic construction of Verlet lists. The conventional approach to computing pairwise interactions using cell lists leads to a large number of unnecessary interparticle distance calculations. In this paper, an algorithm is presented which reduces the number of spurious distance calculations by first sorting the particles along the cell pair axis and then only interacting two particles if their distance along the axis is smaller than the cutoff distance of the interaction. This approach is shown to be more efficient than the conventional approach and similar approaches using smaller cells.