Stochastic Processes and Mean Field Systems Defined by Nonlinear Markov Chains: An Illustration for a Model of Evolutionary Population Dynamics

In physics, there is a growing interest in studying stochastic processes described by evolution equations such as nonlinear master equations and nonlinear Fokker–Planck equations that define the so-called nonlinear Markov processes and are nonlinear with respect to probability densities. In this context, however, relatively little is known about nonlinear Markov processes defined by nonlinear Markov chains. In the present work, we demonstrate explicitly how the nonlinear Markov chain approach can be carried out by addressing a model for evolutionary population dynamics. In line with the nonlinear Markov chain approach, we derive a measure that tells us how attractive it is for a biological entity to evolve towards a particular biological type. Likewise, a measure for the noise level of the evolutionary process is obtained. Both measures are found to be implicitly time dependent. Finally, a simulation scheme for the many-body system corresponding to the Markov chain model is discussed.     

The Ehrenfest model: a path-integral for spin

The Ehrenfest model, formulated as a continuous time Markov chain, is a representation of a spin with length j in a magnetic field. The equivalence is shown by the introduction of a Feynman-Kac functional on the sample paths of the Markov chain. A transformation is used to calculate the functional leading to the solution of a path-integral for spin without reference to coherent states.     

Steady state speed distribution analysis for a combined cellular automaton traffic model

Cellular Automaton （CA） based traffic flow models have been extensively studied due to their effectiveness and simplicity in recent years. This paper develops a discrete time Markov chain （DTMC） analytical framework for a Nagel-Schreckenberg and Fukui Ishibashi combined CA model （W^2H traffic flow model） from microscopic point of view to capture the macroscopic steady state speed distributions. The inter-vehicle spacing Maxkov chain and the steady state speed Markov chain are proved to be irreducible and ergodic. The theoretical speed probability distributions depending on the traffic density and stochastic delay probability are in good accordance with numerical simulations. The derived fundamental diagram of the average speed from theoretical speed distributions is equivalent to the results in the previous work.     

Discrete-time Glauber model as a generalized Ehrenfest urn model

Glauber's continuous-time stochastic model for relaxation of spin systems is “imbedded” in discrete time as a Markov chain. The chain is a generalization of the Ehrenfest urn model as formulated by Hess. Some properties of the chain are discussed.     

The Theory of Individual Based Discrete-Time Processes

A general theory is developed to study individual based models which are discrete in time. We begin by constructing a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are hence intrinsic to the system and can induce qualitative changes to the dynamics predicted from the deterministic map. From the Chapman–Kolmogorov equation for the discrete-time Markov process, we derive the analogues of the Fokker–Planck equation and the Langevin equation, which are routinely employed for continuous time processes. In particular, a stochastic difference equation is derived which accurately reproduces the results found from the Markov chain model. Stochastic corrections to the deterministic map can be quantified by linearizing the fluctuations around the attractor of the map. The proposed scheme is tested on stochastic models which have the logistic and Ricker maps as their deterministic limits.     

基于灰色马尔科夫模型的团簇数量演变预测方法

本文以分子动力学方法模拟液态Si的凝固过程为基础,首次建立基于马尔科夫修正的灰色预测模型对不同团簇的演变过程进行分析,利用该模型对液态Si降温过程中团簇数量的变化进行预测。结果证实应用灰色马尔科夫模型在对团簇结构的演变预测是可行的,采用马尔科夫链预测方法能提高预测结果的准确性,该方法能有效描述凝固过程中团簇的形成和演变的整体变化规律,为研究材料微观结构演变提供了一种新的思路和方法。     

基于灰色马尔科夫模型的团簇数量演变预测方法

本文以分子动力学方法模拟液态Si的凝固过程为基础,首次建立基于马尔科夫修正的灰色预测模型对不同团簇的演变过程进行分析,利用该模型对液态Si降温过程中团簇数量的变化进行预测.结果证实应用灰色马尔科夫模型在对团簇结构的演变预测是可行的,采用马尔科夫链预测方法能提高预测结果的准确性,该方法能有效描述凝固过程中团簇的形成和演变的整体变化规律,为研究材料微观结构演变提供了一种新的思路和方法 .     

Optimal instruments and models for noisy chaos

Analysis of finite, noisy time series data leads to modern statistical inference methods. Here we adapt Bayesian inference for applied symbolic dynamics. We show that reconciling Kolmogorov's maximum-entropy partition with the methods of Bayesian model selection requires the use of two separate optimizations. First, instrument design produces a maximum-entropy symbolic representation of time series data. Second, Bayesian model comparison with a uniform prior selects a minimum-entropy model, with respect to the considered Markov chain orders, of the symbolic data. We illustrate these steps using a binary partition of time series data from the logistic and Henon maps as well as the R?ssler and Lorenz attractors with dynamical noise. In each case we demonstrate the inference of effectively generating partitions and kth-order Markov chain models.     

Evolution analysis of the states of the EZ model

Based on suitable choice of states, this paper studies the stability of the equilibrium state of the EZ model by regarding the evolution of the EZ model as a Markov chain and by showing that the Markov chain is ergodic. The Markov analysis is applied to the EZ model with small number of agents, the exact equilibrium state for N = 5 and numerical results for N=18 are obtained.     

Markov chains theory for scale-free networks

This paper proposes a Markov chain method to predict the growth dynamics of the individual nodes in scale-free networks, and uses this to calculate numerically the degree distribution. We first find that the degree evolution of a node in the BA model is a nonhomogeneous Markov chain. An efficient algorithm to calculate the degree distribution is developed by the theory of Markov chains. The numerical results for the BA model are consistent with those of the analytical approach. A directed network with the logarithmic growth is introduced. The algorithm is applied to calculate the degree distribution for the model. The numerical results show that the system self-organizes into a scale-free network.     

Algebraic decay in self-similar Markov chains

A continuous-time Markov chain is used to model motion in the neighborhood of a critical invariant circle for a Hamiltonian map. States in the infinite chain represent successive rational approximants to the frequency of the invariant circle. For the case of a noble frequency, the chain is self-similar and the nonlinear integral equation for the first passage time distribution is solved exactly. The asymptotic distribution is a power law times a function periodic in the logarithm of the time. For parameters relevant to the critical noble circle, the decay proceeds ast ^–4.05.     

Polymers and random graphs

We establish a precise connection between gelation of polymers in Lushnikov's model and the emergence of the giant component in random graph theory. This is achieved by defining a modified version of the Erdös-Rényi process; when contracting to a polymer state space, this process becomes a discrete-time Markov chain embedded in Lushnikov's process. The asymptotic distribution of the number of transitions in Lushnikov's model is studied. A criterion for a general Markov chain to retain the Markov property under the grouping of states is derived. We obtain a noncombinatorial proof of a theorem of Erdös-Rényi type.     

A time series analysis and a non-homogeneous Poisson model with multiple change-points applied to acoustic data

High levels of the so-called community noise may produce hazardous effect on the health of a population exposed to them for large periods of time. Hence, the study of the behaviour of those noise measurements is very important. In this work we analyse that in terms of the probability of exceeding a given threshold level a certain number of times in a time interval of interest. Since the datasets considered contain missing measurements, we use a time series model to estimate the missing values and complete the datasets. Once the data is complete, we use a non-homogeneous Poisson model with multiple change-points to estimate the probability of interest. Estimation of the parameters of the models are made using the usual time series methodology as well as the Bayesian point of view via Markov chain Monte Carlo algorithms. The models are applied to data obtained from two measuring sites in Messina, Italy.     

Vulnerability to paroxysmal oscillations in delayed neural networks: a basis for nocturnal frontal lobe epilepsy?

Resonance can occur in bistable dynamical systems due to the interplay between noise and delay (τ) in the absence of a periodic input. We investigate resonance in a two-neuron model with mutual time-delayed inhibitory feedback. For appropriate choices of the parameters and inputs three fixed-point attractors co-exist: two are stable and one is unstable. In the absence of noise, delay-induced transient oscillations (referred to herein as DITOs) arise whenever the initial function is tuned sufficiently close to the unstable fixed-point. In the presence of noisy perturbations, DITOs arise spontaneously. Since the correlation time for the stationary dynamics is ～τ, we approximated a higher order Markov process by a three-state Markov chain model by rescaling time as t?→?2sτ, identifying the states based on whether the sub-intervals were completely confined to one basin of attraction (the two stable attractors) or straddled the separatrix, and then determining the transition probability matrix empirically. The resultant Markov chain model captured the switching behaviors including the statistical properties of the DITOs. Our observations indicate that time-delayed and noisy bistable dynamical systems are prone to generate DITOs as switches between the two attractors occur. Bistable systems arise transiently in situations when one attractor is gradually replaced by another. This may explain, for example, why seizures in certain epileptic syndromes tend to occur as sleep stages change.     

The Convergence of Markov Chain Monte Carlo Methods: From the Metropolis Method to Hamiltonian Monte Carlo

From its inception in the 1950s to the modern frontiers of applied statistics, Markov chain Monte Carlo has been one of the most ubiquitous and successful methods in statistical computing. The development of the method in that time has been fueled by not only increasingly difficult problems but also novel techniques adopted from physics. Here, the history of Markov chain Monte Carlo is reviewed from its inception with the Metropolis method to the contemporary state‐of‐the‐art in Hamiltonian Monte Carlo, focusing on the evolving interplay between the statistical and physical perspectives of the method.     

The Optimal Sink and the Best Source in a Markov Chain

It is well known that the distributions of hitting times in Markov chains are quite irregular, unless the limit as time tends to infinity is considered. We show that nevertheless for a typical finite irreducible Markov chain and for nondegenerate initial distributions the tails of the distributions of the hitting times for the states of a Markov chain can be ordered, i.e., they do not overlap after a certain finite moment of time. If one considers instead each state of a Markov chain as a source rather than a sink then again the states can generically be ordered according to their efficiency. The mechanisms underlying these two orderings are essentially different though. Our results can be used, e.g., for a choice of the initial distribution in numerical experiments with the fastest convergence to equilibrium/stationary distribution, for characterization of the elements of a dynamical network according to their ability to absorb and transmit the substance (“information”) that is circulated over the network, for determining optimal stopping moments (stopping signals/words) when dealing with sequences of symbols, etc.     

模拟回火马尔可夫链蒙特卡罗全波形分析方法

针对传统的全波形分析方法不能快速自动处理全波形数据的缺点,提出了一种模拟回火马尔可夫链蒙特卡罗全波形分析法,用于求解全波形数据中的波峰数和峰值位置等参量.该方法采用Metropolis更新策略求解波峰数量和噪声两个参量,以达到快速求解的目的;而峰值位置和波峰幅值则采用改进的模拟回火策略求解,通过添加的主动干预回火步骤实现对参量更新过程的有效探测,以满足对速度或运算收敛性的要求.模拟回火马尔可夫链蒙特卡罗全波形分析方法以马尔可夫算法为基础,仍保持马氏链的收敛性,从而保证本方法具有良好的鲁棒性,实现对全波形数据的自动化处理.     

Degree distribution of a new model for evolving networks

We propose and study an evolving network model with both preferential and random attachments of new links, incorporating the addition of new nodes, new links, and the removal of links. We first show that the degree evolution of a node follows a nonhomogeneous Markov chain. Based on the concept of Markov chain, we provide the exact solution of the degree distribution of this model and show that the model can generate scale-free evolving network.     

A Simple Discrete Model of Brownian Motors: Time-periodic Markov Chains

In this paper, we consider periodically inhomogeneous Markov chains, which can be regarded as a simple version of physical model—Brownian motors. We introduce for them the concepts of periodical reversibility, detailed balance, entropy production rate and circulation distribution. We prove the equivalence of the following statements: The time-periodic Markov chain is periodically reversible; It is in detailed balance; Kolmogorov's cycle condition is satisfied; Its entropy production rate vanishes; Every circuit and its reversed circuit have the same circulation weight. Hence, in our model of Markov chains, the directed transport phenomenon of Brownian motors, i.e. the existence of net circulation, can occur only in nonequilibrium and irreversible systems. Moreover, we verify the large deviation property and the Gallavotti-Cohen fluctuation theorem of sample entropy production rates of the Markov chain.     

Metastates in Mean-Field Models with Random External Fields Generated by Markov Chains

We extend the construction by Külske and Iacobelli of metastates in finite-state mean-field models in independent disorder to situations where the local disorder terms are a sample of an external ergodic Markov chain in equilibrium. We show that for non-degenerate Markov chains, the structure of the theorems is analogous to the case of i.i.d. variables when the limiting weights in the metastate are expressed with the aid of a CLT for the occupation time measure of the chain.