Chaos from nonlinear Markov processes: Why the whole is different from the sum of its parts

Nonlinear Markov processes have been frequently used to address bifurcations and multistability in equilibrium and non-equilibrium many-body systems. However, our understanding of the range of phenomena produced by nonlinear Markov processes is still in its infancy. We demonstrate that in addition to bifurcations and multistability nonlinear Markov processes can exhibit another key phenomena well known in the realm of nonlinear physics: chaos. It is argued that chaotically evolving process probabilities are a generic feature of many-body systems exhibiting nonlinear Markov processes even if the isolated subsystems do not exhibit chaos. That is, when considering a nonlinear Markov process as an entity of its own type, then the nonlinear Markov process in general is qualitatively different from its constituent subprocesses, which reflects that the many-body system as a whole is different from the sum of its parts.     

Nonlinear Markov processes: Deterministic case

Deterministic Markov processes that exhibit nonlinear transition mechanisms for probability densities are studied. In this context, the following issues are addressed: Markov property, conditional probability densities, propagation of probability densities, multistability in terms of multiple stationary distributions, stability analysis of stationary distributions, and basin of attraction of stationary distribution.     

Hamiltonian and Lagrangian for the Trajectory of the Empirical Distribution and the Empirical Measure of Markov Processes

We compute the Hamiltonian and Lagrangian associated to the large deviations of the trajectory of the empirical distribution for independent Markov processes, and of the empirical measure for translation invariant interacting Markov processes. We treat both the case of jump processes (continuous-time Markov chains and interacting particle systems) as well as diffusion processes. For diffusion processes, the Lagrangian is a quadratic form of the deviation of the trajectory from the solution of the Kolmogorov forward equation. In all cases, the Lagrangian can be interpreted as a relative entropy or relative entropy density per unit time.     

Stochastic Processes and Mean Field Systems Defined by Nonlinear Markov Chains: An Illustration for a Model of Evolutionary Population Dynamics

In physics, there is a growing interest in studying stochastic processes described by evolution equations such as nonlinear master equations and nonlinear Fokker–Planck equations that define the so-called nonlinear Markov processes and are nonlinear with respect to probability densities. In this context, however, relatively little is known about nonlinear Markov processes defined by nonlinear Markov chains. In the present work, we demonstrate explicitly how the nonlinear Markov chain approach can be carried out by addressing a model for evolutionary population dynamics. In line with the nonlinear Markov chain approach, we derive a measure that tells us how attractive it is for a biological entity to evolve towards a particular biological type. Likewise, a measure for the noise level of the evolutionary process is obtained. Both measures are found to be implicitly time dependent. Finally, a simulation scheme for the many-body system corresponding to the Markov chain model is discussed.     

Self-Duality of Markov Processes and Intertwining Functions

We present a theorem which elucidates the connection between self-duality of Markov processes and representation theory of Lie algebras. In particular, we identify sufficient conditions such that the intertwining function between two representations of a certain Lie algebra is the self-duality function of a (Markov) operator. In concrete terms, the two representations are associated to two operators in interwining relation. The self-dual operator, which arise from an appropriate symmetric linear combination of them, is the generator of a Markov process. The theorem is applied to a series of examples, including Markov processes with a discrete state space (e.g. interacting particle systems) and Markov processes with continuous state space (e.g. diffusion processes). In the examples we use explicit representations of Lie algebras that are unitarily equivalent. As a consequence, in the discrete setting self-duality functions are given by orthogonal polynomials whereas in the continuous context they are Bessel functions.     

Accelerated Stochastic Simulation of Large Chemical Systems

For efficient simulation of chemical systems with large number of reactions, we report a fast and exact algorithm for direct simulation of chemical discrete Markov processes. The approach adopts the scheme of organizing the reactions into hierarchical groups. By generating a random number, the selection of the next reaction that actually occurs is accomplished by a few successive selections in the hierarchical groups. The algorithm which is suited for simulating systems with large number of reactions is much faster than the direct method or the optimized direct method. For a demonstration of its efficiency, the accelerated algorithm is applied to simulate the reaction-diffusion Brusselator model on a discretized space.     

Quantum stochastic processes

We first define a class of processes which we call regular quantum Markov processes. We next prove some basic results concerning such processes. A method is given for constructing quantum Markov processes using transition amplitude kernels. Finally we show that the Feynman path integral formalism can be clarified by approximating it with a quantum stochastic process.     

A definition of metastability for Markov processes with detailed balance

A definition of metastable states applicable to arbitrary finite state Markov processes satisfying detailed balance is discussed. In particular, we identify a crucial condition that distinguishes genuine metastable states from other types of slowly decaying modes and which leads to properties similar to those postulated in the restricted ensemble approach [1]. The intuitive physical meaning of this condition is simply that the total equilibrium probability of finding the system in the metastable state is negligible. As a concrete application of our formalism we present preliminary results on a 2D kinetic Ising model.     

The maximal process of nonlinear shot noise

In the nonlinear shot noise system-model shots’ statistics are governed by general Poisson processes, and shots’ decay-dynamics are governed by general nonlinear differential equations. In this research we consider a nonlinear shot noise system and explore the process tracking, along time, the system’s maximal shot magnitude. This ‘maximal process’ is a stationary Markov process following a decay-surge evolution; it is highly robust, and it is capable of displaying both a wide spectrum of statistical behaviors and a rich variety of random decay-surge sample-path trajectories. A comprehensive analysis of the maximal process is conducted, including its Markovian structure, its decay-surge structure, and its correlation structure. All results are obtained analytically and in closed-form.     

A Variational Principle for Markov Processes

In this note, we first present a result concerning a variational principle for general Markov processes. Then we apply it to spin particle systems to obtain a full form of a variational principle characterizing the stationary Markov laws of the systems. A related extreme decomposition for any stationary distribution of such Markov systems is also given.     

First passage time and extremum properties of Markov and independent processes

It was shown by Newell in 1962 that the extreme value and first passage time distributions of various types of common Markov processes asymptotically approach those for independent random variables. In view of the great simplification this occasions in the calculation of a number of important properties of Markov processes, it is clearly of interest to determine in some detail the conditions on both the time and space variables under which this equivalence holds. In this paper we investigate and establish these conditions for Markov processes described by the Fokker-Planck equation and express them in simple analytic forms which are directly related to the coefficients of the Fokker-Planck equation. To demonstrate the usefulness of these conditions, we apply them to two representative examples of Fokker-Planck equations, the Ornstein-Uhlenbeck process and the Montroll-Shuler model for harmonic oscillator dissociation. It is shown very clearly in these examples that the extreme value and first passage time     

Transition events fluctuations in jump markovian dynamics. A general approach to Šolc's problem

We introduce a new formalism for computing the moments of transition events for nonhomogeneous Markov jump processes. Our method is applied directly to the master equation and does not involve the use of diffusion approximation. The general theory is applied to produce exact expressions for means and dispersions. For time homogeneous Markov processes with a finite number of connected states we are able to prove that both means and dispersions asymptotically increase linearly in time.     

On the equivalence between Bernoulli dynamical systems and stochastic Markov processes

We extend to Bernoulli systems the explicit construction (elaborated previously for the baker transformation) of non-unitary, invertible transformations Λ, which associate Markovian processes admitting an $\text{H}$-theorem with the unitary dynamical group, through a similarity relation. We characterize the symmetries of the Bernoulli system as well as those of the associated Markov processes and provide examples of symmetry breaking under the passage, through a Λ transformation, from Bernoulli systems to stochastic Markov processes.     

A mini-tutorial on measure-valued Markov processes and nonlinear martingale problems—As a reply to McCauley's comment

One goal of this mini-tutorial is to provide an introduction into the theory of measure-valued Markov processes and nonlinear martingales defined by strongly nonlinear Fokker-Planck equations and to discuss the physical relevance of the associated processes. Another goal is to reply to McCauley's comment on T.D. Frank [Physica A 331, 391 (2004)]. The tutorial addresses in detail two approaches found in physics and mathematics. The first approach exploits a mapping between linear and nonlinear Fokker-Planck equations. The second approach exploits martingale theory. Several examples of Markov processes and martingales in quantum mechanical, nonextensive, and self-organizing systems defined by nonlinear Fokker-Planck equations are discussed.     

Duality in Interacting Particle Systems and Boson Representation

In the context of Markov processes, we show a new scheme to derive dual processes and a duality function based on a boson representation. This scheme is applicable to a case in which a generator is expressed by boson creation and annihilation operators. For some stochastic processes, duality relations have been known, which connect continuous time Markov processes with discrete state space and those with continuous state space. We clarify that using a generating function approach and the Doi-Peliti method, a birth-death process (or discrete random walk model) is naturally connected to a differential equation with continuous variables, which would be interpreted as a dual Markov process. The key point in the derivation is to use bosonic coherent states as a bra state, instead of a conventional projection state. As examples, we apply the scheme to a simple birth-coagulation process and a Brownian momentum process. The generator of the Brownian momentum process is written by elements of the SU(1,1) algebra, and using a boson realization of SU(1,1) we show that the same scheme is available.     

The Viterbi Algorithm for Models of Hidden Markov Processes with the Unknown Moment of Appearance of Parameter Jump

The methods of the theory of optimal nonlinear filtering of the Markov processes is used to develop the Viterbi algorithm for obtaining optimal estimates of a sequence of hidden states in the model of discrete-value Markov processes generalized to the case of jump-like changing parameters with an unknown time of the jump appearance. The results of numerical simulation of the algorithm performance are given. __________ Translated from Izvestiya Vysshikh Uchebnykh Zavedenii, Radiofizika, Vol. 48, No. 4, pp. 358–366, April 2005.     

Estimation of Markov Sequences with Jump-Like Variation of Parameters by the Interpolation Method

Using the methods of the theory of optimal nonlinear filtering, we develop an algorithm for obtaining optimal estimates of the sequence of hidden states of discrete-valued Markov processes with abruptly changing parameters at unknown time. Optimal estimates of the states of Markov processes and of the time of appearance of an abrupt change in parameters are obtained as a result of interpolation by processing the entire observation sequence. The results of simulation of the algorithm work are presented. __________ Translated from Izvestiya Vysshikh Uchebnykh Zavedenii, Radiofizika, Vol. 48, No. 7, pp. 628–639, July 2005.     

Permutation approach to finite-alphabet stationary stochastic processes based on the duality between values and orderings

The duality between values and orderings is a powerful tool to discuss relationships between various information-theoretic measures and their permutation analogues for discrete-time finite-alphabet stationary stochastic processes (SSPs). Applying it to output processes of hidden Markov models with ergodic internal processes, we have shown in our previous work that the excess entropy and the transfer entropy rate coincide with their permutation analogues. In this paper, we discuss two permutation characterizations of the two measures for general ergodic SSPs not necessarily having the Markov property assumed in our previous work. In the first approach, we show that the excess entropy and the transfer entropy rate of an ergodic SSP can be obtained as the limits of permutation analogues of them for the N-th order approximation by hidden Markov models, respectively. In the second approach, we employ the modified permutation partition of the set of words which considers equalities of symbols in addition to permutations of words. We show that the excess entropy and the transfer entropy rate of an ergodic SSP are equal to their modified permutation analogues, respectively.