Markov processes follow from the principle of maximum caliber

Markov models are widely used to describe stochastic dynamics. Here, we show that Markov models follow directly from the dynamical principle of maximum caliber (Max Cal). Max Cal is a method of deriving dynamical models based on maximizing the path entropy subject to dynamical constraints. We give three different cases. First, we show that if constraints (or data) are given in the form of singlet statistics (average occupation probabilities), then maximizing the caliber predicts a time-independent process that is modeled by identical, independently distributed random variables. Second, we show that if constraints are given in the form of sequential pairwise statistics, then maximizing the caliber dictates that the kinetic process will be Markovian with a uniform initial distribution. Third, if the initial distribution is known and is not uniform we show that the only process that maximizes the path entropy is still the Markov process. We give an example of how Max Cal can be used to discriminate between different dynamical models given data.     

非平衡定态化学反应体系对细致平衡偏离的不可逆性之随机热力学量度

在随机描述的层次上,成功地构筑了非平衡定态化学反应体系对细致平衡偏离的不可逆性之随机热力学判据.基于Fokker-Planck方程建立了连续Markov过程系统的随机熵平衡方程,发现在随机态空间中随机熵的时间变率亦可分为源项和流项两部分贡献.对于化学反应体系,作为态空间源项的随机熵产生可作为偏离细致平衡不可逆性的合适度量,此泛函量的零值标志着细致平衡.进一步借助按系统广度量的Ω展开法,通过对随机力及共轭的随机流的分析揭示态空间中的随机熵产生仅源于状态对细致平衡的偏离,并主要来自非Poisson涨落的贡献,因而可作为随机热力学量去判别和量度化学反应体系对细致平衡的偏离.     

Localization–delocalization phenomena in a cyclic box

The localization–delocalization of a particle in a cyclic box is studied by examination of its Shannon information entropy and standard deviation. These results are compared to the particle in a box model, in ground and also in excited states. We show how a cyclic symmetry imposes that the ground state is more delocalized in the cyclic box as compared to the particle in a box. However, excited states in both models exhibit the same localization. The differences between the Shannon entropy and the standard deviation are discussed and the analysis is then extended to consider multiple particles in both models.     

Foldamer dynamics expressed via Markov state models. II. State space decomposition

The structural landscape of poly-phenylacetylene (pPA), otherwise known as m-phenylene ethynylene oligomers, has been shown to consist of a very diverse set of conformations, including helices, turns, and knots. Defining a state space decomposition to classify these conformations into easily identifiable states is an important step in understanding the dynamics in relation to Markov state models. We define the state decomposition of pPA oligomers in terms of the sequence of discretized dihedral angles between adjacent phenyl rings along the oligomer backbone. Furthermore, we derive in mathematical detail an approach to further reduce the number of states by grouping symmetrically equivalent states into a single parent state. A more challenging problem requires a formal definition for knotted states in the structural landscape. Assuming that the oligomer chain can only cross the ideal helix path once, we propose a technique to define a knotted state derived from a helical state determined by the position along the helical nucleus where the chain crosses the ideal helix path. Several examples of helical states and knotted states from the pPA 12-mer illustrate the principles outlined in this article.     

Measurement of nonequilibrium entropy from space-time thermodynamic integration

The entropy of a system transiently driven out of equilibrium by a time-inhomogeneous stochastic dynamics is first expressed as a transient response function generalizing the nonlinear Kawasaki-Crooks response. This function is then reformulated into three statistical averages defined over ensembles of nonequilibrium trajectories. The first average corresponds to a space-time thermodynamic perturbation relation, while the two following ones correspond to space-time thermodynamic integration relations. Provided that trajectories are initiated starting from a distribution of states that is analytically known, the ensemble averages are computationally amenable to Markov chain Monte Carlo methods. The relevance of importance sampling in path ensembles is confirmed in practice by computing the nonequilibrium entropy of a driven toy system. We finally study a situation where the dynamics produces entropy. In this case, we observe that space-time thermodynamic integration still yields converged estimates, while space-time thermodynamic perturbation turns out to converge very slowly.     

A Bayesian method for construction of Markov models to describe dynamics on various time-scales

The dynamics of many biological processes of interest, such as the folding of a protein, are slow and complicated enough that a single molecular dynamics simulation trajectory of the entire process is difficult to obtain in any reasonable amount of time. Moreover, one such simulation may not be sufficient to develop an understanding of the mechanism of the process, and multiple simulations may be necessary. One approach to circumvent this computational barrier is the use of Markov state models. These models are useful because they can be constructed using data from a large number of shorter simulations instead of a single long simulation. This paper presents a new Bayesian method for the construction of Markov models from simulation data. A Markov model is specified by (τ,P,T), where τ is the mesoscopic time step, P is a partition of configuration space into mesostates, and T is an N(P)×N(P) transition rate matrix for transitions between the mesostates in one mesoscopic time step, where N(P) is the number of mesostates in P. The method presented here is different from previous Bayesian methods in several ways. (1) The method uses Bayesian analysis to determine the partition as well as the transition probabilities. (2) The method allows the construction of a Markov model for any chosen mesoscopic time-scale τ. (3) It constructs Markov models for which the diagonal elements of T are all equal to or greater than 0.5. Such a model will be called a "consistent mesoscopic Markov model" (CMMM). Such models have important advantages for providing an understanding of the dynamics on a mesoscopic time-scale. The Bayesian method uses simulation data to find a posterior probability distribution for (P,T) for any chosen τ. This distribution can be regarded as the Bayesian probability that the kinetics observed in the atomistic simulation data on the mesoscopic time-scale τ was generated by the CMMM specified by (P,T). An optimization algorithm is used to find the most probable CMMM for the chosen mesoscopic time step. We applied this method of Markov model construction to several toy systems (random walks in one and two dimensions) as well as the dynamics of alanine dipeptide in water. The resulting Markov state models were indeed successful in capturing the dynamics of our test systems on a variety of mesoscopic time-scales.     

A derivation of the master equation from path entropy maximization

The master equation and, more generally, Markov processes are routinely used as models for stochastic processes. They are often justified on the basis of randomization and coarse-graining assumptions. Here instead, we derive nth-order Markov processes and the master equation as unique solutions to an inverse problem. We find that when constraints are not enough to uniquely determine the stochastic model, an nth-order Markov process emerges as the unique maximum entropy solution to this otherwise underdetermined problem. This gives a rigorous alternative for justifying such models while providing a systematic recipe for generalizing widely accepted stochastic models usually assumed to follow from the first principles.     

Building Markov state models along pathways to determine free energies and rates of transitions

An efficient method is proposed for building Markov models with discrete states able to accurately describe the slow relaxation of a complex system with two stable conformations. First, the reaction pathway described by a set of collective variables between the two stable states is determined using the string method with swarms of trajectories. Then, short trajectories are initiated at different points along this pathway to build the state-to-state transition probability matrix. It is shown, using a model system, how this strategy makes it possible to use trajectories that are significantly shorter than the slowest relaxation time to efficiently build a reliable and accurate Markov model. Extensions of the method to multiple pathways, as well as some common pitfalls arising from poorly relaxed paths or an inappropriate choice of collective variables, are illustrated and discussed.     

How the nature of an observation affects single-trajectory entropies

Projection of a Markov process with constant rates of transition to a small number of observable aggregated states can result in complex kinetics with memory. Here, we define the entropy production along a single sequence of aggregated states and show that it obeys detailed and integral fluctuation theorems. More importantly, we prove that projection shifts the distribution of entropy production over the ensemble of paths for a nonequilibrium process toward one characteristic of a system at equilibrium. This statement represents an analog of the second law of thermodynamics for path-dependent entropies and thus a new form of constraint of irreversible systems. Numeric examples are presented to illustrate these ideas.     

An efficient, path-independent method for free-energy calculations

Classical free-energy methods depend on the definition of physical or nonphysical integration paths to calculate free-energy differences between states. This procedure can be problematic and computationally expensive when the states of interest do not overlap and are far apart in phase space. Here we introduce a novel method to calculate free-energy differences that is path-independent by transforming each end state into a reference state in which the vibrational entropy is the sole component of the total entropy, thus allowing direct computation of the relative free energy. We apply the method to calculate side-chain entropies of a beta-hairpin-forming peptide in a variety of backbone conformations, demonstrating its importance in determining structural propensities. We find that low-free-energy conformations achieve their stability through optimal trade off between enthalpic gains due to favorable interatomic interactions and entropic losses incurred by the same.     

Automatic discovery of metastable states for the construction of Markov models of macromolecular conformational dynamics

To meet the challenge of modeling the conformational dynamics of biological macromolecules over long time scales, much recent effort has been devoted to constructing stochastic kinetic models, often in the form of discrete-state Markov models, from short molecular dynamics simulations. To construct useful models that faithfully represent dynamics at the time scales of interest, it is necessary to decompose configuration space into a set of kinetically metastable states. Previous attempts to define these states have relied upon either prior knowledge of the slow degrees of freedom or on the application of conformational clustering techniques which assume that conformationally distinct clusters are also kinetically distinct. Here, we present a first version of an automatic algorithm for the discovery of kinetically metastable states that is generally applicable to solvated macromolecules. Given molecular dynamics trajectories initiated from a well-defined starting distribution, the algorithm discovers long lived, kinetically metastable states through successive iterations of partitioning and aggregating conformation space into kinetically related regions. The authors apply this method to three peptides in explicit solvent-terminally blocked alanine, the 21-residue helical F(s) peptide, and the engineered 12-residue beta-hairpin trpzip2-to assess its ability to generate physically meaningful states and faithful kinetic models.     

An efficient method for computing excess free energy of liquid

We present a new theoretical method for efficient calculation of free energy of liquid. This interaction entropy method allows one to compute entropy and free energy of liquid from standard single step MD (molecular dynamics) simulation directly in liquid state without the need to perform MD simulations at many intermediate states as required in thermodynamic integration or free energy perturbation methods. In this new approach, one only needs to evaluate the interaction energy of a single (fixed) liquid molecule with the rest of liquid molecules as a function of time from a standard MD simulation of liquid and the fluctuation of distribution of this interaction energy is then used to calculate the interaction entropy of the liquid. Explicit theoretical derivation of this interaction entropy approach is provided and numerical calculations for the benchmark liquid water system were carried out using three different water models. Numerical analysis of the result was performed and comparison of the computational result with experimental data and other theoretical results were provided. Excellent agreement of calculated free energies with the experimental data using TIP4P model is obtained for liquid water.     

Calculation of entropy with computer simulation methods

A new approximate method for calculating entropy with computer simulation methods is proposed. The sampling probability of an equilibrium configuration is calculated by means of the frequency of occurrence of what we call here local states (related to the state of the particle and its neighbors); hence entropy can be calculated as well. The method is applied to the Metropolis' Monte Carlo simulation for the case of the square Ising lattice. The results are in very good agreement with theory even close to the critical temperature. The method is used also to calculate non-equilibrium entropy.     

Hidden Conformations in Aspergillus niger Monoamine Oxidase are Key for Catalytic Efficiency

Enzymes exist as an ensemble of conformational states, whose populations can be shifted by substrate binding, allosteric interactions, but also by introducing mutations to their sequence. Tuning the populations of the enzyme conformational states through mutation enables evolution towards novel activity. Herein, Markov state models are used to unveil hidden conformational states of monoamine oxidase from Aspergillus niger (MAO‐N). These hidden conformations, not previously observed by any other technique, play a crucial role in substrate binding and enzyme activity. This reveals how distal mutations regulate MAO‐N activity by stabilizing these hidden, catalytically important conformational states, but also by modulating the communication pathway between both MAO‐N subunits.     

Importance of chirality and reduced flexibility of protein side chains: a study with square and tetrahedral lattice models

Side chains of amino acid residues are the determining factor that distinguishes proteins from other unstable chain polymers. In simple models they are often represented implicitly (e.g., by spin states) or simplified as one atom. Here we study side chain effects using two-dimensional square lattice and three-dimensional tetrahedral lattice models, with explicitly constructed side chains formed by two atoms of different chirality and flexibility. We distinguish effects due to chirality and effects due to side chain flexibilities, since residues in proteins are L residues, and their side chains adopt different rotameric states. For short chains, we enumerate exhaustively all possible conformations. For long chains, we sample effectively rare events such as compact conformations and obtain complete pictures of ensemble properties of conformations of these models at all compactness region. This is made possible by using sequential Monte Carlo techniques based on chain growth method. Our results show that both chirality and reduced side chain flexibility lower the folding entropy significantly for globally compact conformations, suggesting that they are important properties of residues to ensure fast folding and stable native structure. This corresponds well with our finding that natural amino acid residues have reduced effective flexibility, as evidenced by statistical analysis of rotamer libraries and side chain rotatable bonds. We further develop a method calculating the exact side chain entropy for a given backbone structure. We show that simple rotamer counting underestimates side chain entropy significantly for both extended and near maximally compact conformations. We find that side chain entropy does not always correlate well with main chain packing. With explicit side chains, extended backbones do not have the largest side chain entropy. Among compact backbones with maximum side chain entropy, helical structures emerge as the dominating configurations. Our results suggest that side chain entropy may be an important factor contributing to the formation of alpha helices for compact conformations.     

The Coordinate Reaction Model: An Obstacle to Interpreting the Emergence of Chemical Complexity

The way chemical transformations are described by models based on microscopic reversibility does not take into account the irreversibility of natural processes, and therefore, in complex chemical networks working in open systems, misunderstandings may arise about the origin and causes of the stability of non-equilibrium stationary states, and general constraints on evolution in systems that are far from equilibrium. In order to be correctly simulated and understood, the chemical behavior of complex systems requires time-dependent models, otherwise the irreversibility of natural phenomena is overlooked. Micro reversible models based on the reaction-coordinate model are time invariant and are therefore unable to explain the evolution of open dissipative systems. The important points necessary for improving the modeling and simulations of complex chemical systems are: a) understanding the physical potential related to the entropy production rate, which is in general an inexact differential of a state function, and b) the interpretation and application of the so-called general evolution criterion (GEC), which is the general thermodynamic constraint for the evolution of dissipative chemical systems.     

Benchmark values of Shannon entropy for spherically confined hydrogen atom

The information‐theoretic measure of confined hydrogen atom has been investigated extensively in the literature. However, most of them were focused on the ground state and accurate values of information entropies, such as Shannon entropy, for confined hydrogen are still not determined. In this work, we establish the benchmark results of the Shannon entropy for confined hydrogen atom in a spherical impenetrable sphere, in both position and momentum spaces. This is done by examining the bound state energies, the normalization of wave functions, and the scaling property with respect to isoelectronic hydrogenic ions. The angular and radial parts of Shannon entropy in two conjugate spaces are provided in detail for both free and confined hydrogen atom in ground and several excited states. The entropies in position space decrease logarithmically with decreasing the size of confinement, while those in momentum space increase logarithmically. The Shannon entropy sum, however, approaches to finite values when the confinement radius closes to zero. It is also found that the Shannon entropy sum shares same trend for states with similar density distributions. Variations of entropy for nodeless bound states are significantly distinct form those owning nodes when changing the confinement radius.     

Bacteriorhodopsin photocycle kinetics analyzed by the maximum entropy method

A maximum entropy method (MEM) was developed for the study of the bacteriorhodopsin photocycle kinetics. The method can be applied directly to experimental kinetic absorption data without any assumption for the number of the intermediate states taking part in the photocycle. Though this method does not give a specific kinetics, its result is very useful for selection between possible photocycle kinetics. Using simulated data, it is shown that MEM gives correct results for the number of the intermediate states and the amplitude distributions around the characteristic lifetimes. Analyzing experimental absorption data at five different wavelengths, MEM gives seven or eight characteristic lifetimes, which means that at least so many distinct intermediate states exist during the photocycle. Many possible photocycle kinetic models were studied and compared with the MEM result. The best agreement was found with a branching photocycle model of eight intermediate states (K, L, M(1), M(2), M(3), M(4), N, O). The branching occurs at the L intermediate state (M(1) and M(2) being in one branch and M(3) and M(4) in the other branch), but at high pH it occurs already at the K state.