On the Critical Dimension and AC Entropy for Markov Odometers

Markov odometers are natural models for non-homogeneous Markov chains, and are natural generalisations of infinite product measures. We show how to calculate the critical dimension of these measures: this is an invariant which describes the asymptotic growth rate of sums of Radon-Nikodym derivatives. This interesting invariant appears to give a kind of entropy for non-singular odometer actions. The techniques require a law of large numbers for inhomogeneous Markov chains.     

Ergodic Control of a Singularly Perturbed Markov Process in Discrete Time with General State and Compact Action Spaces

Ergodic control of singularly perturbed Markov chains with general state and compact action spaces is considered. A new method is given for characterization of the limit of invariant measures, for perturbed chains, when the perturbation parameter goes to zero. It is also demonstrated that the limit control principle is satisfied under natural ergodicity assumptions about controlled Markov chains. These assumptions allow for the presence of transient states, a situation that has not been considered in the literature before in the context of control of singularly perturbed Markov processes with long-run-average cost functionals. Accepted 3 December 1996     

Random invariant measures for Markov chains,and independent particle systems

Summary Let P be the transition operator for a discrete time Markov chain on a space S. The object of the paper is to study the class of random measures on S which have the property that MP=M in distribution. These will be called random invariant measures for P. In particular, it is shown that MP=M in distribution implies MP=M a.s. for various classes of chains, including aperiodic Harris recurrent chains and aperiodic irreducible random walks. Some of this is done by exploiting the relationship between random invariant measures and entrance laws. These results are then applied to study the invariant probability measures for particle systems in which particles move independently in discrete time according to P. Finally, it is conjectured that every Markov chain which has a random invariant measure also has a deterministic invariant measure.Research supported in part by N.S.F. Grant No. MCS 77-02121     

Censoring,Factorizations, and Spectral Analysis for Transition Matrices with Block-Repeating Entries

In this paper, we use the Markov chain censoring technique to study infinite state Markov chains whose transition matrices possess block-repeating entries. We demonstrate that a number of important probabilistic measures are invariant under censoring. Informally speaking, these measures involve first passage times or expected numbers of visits to certain levels where other levels are taboo; they are closely related to the so-called fundamental matrix of the Markov chain which is also studied here. Factorization theorems for the characteristic equation of the blocks of the transition matrix are obtained. Necessary and sufficient conditions are derived for such a Markov chain to be positive recurrent, null recurrent, or transient based either on spectral analysis, or on a property of the fundamental matrix. Explicit expressions are obtained for key probabilistic measures, including the stationary probability vector and the fundamental matrix, which could be potentially used to develop various recursive algorithms for computing these measures.     

双无限随机环境中的常返马氏链

对双无限随机环境中的马氏链,给出了常返的两种可能的定义,讨论了它们间的联系和基本性质,给出了状态或链为常返的判断准则.讨论了双无限随机环境中马氏链的不变测度的存在性,首次给出了双无限随机环境中马氏链的正常返及零常返的概念,并讨论了其相关性质.特别地,应用不变函数的性质,给出了状态具有正常返性或零常返性的判断准则.     

The Properties and Applications of Invariable Function for Markov Chains in Random Environments

In this paper, various concepts of recurrence and transience are introduced into the research field of Markov chains in random environments, and the concepts and properties of invariant function for Markov chains in random environments are investigated. By using those properties, we obtain a criterion for the state to be recurrent or transient.     

CONSTRUCTIONS OF THE INVARIANT SETS AND INVARIANT MEASURES

In this paper, we will discuss the constructiOn problems about the invariant sets and invariant measures of continues maps~ which map complexes into themselves, using simplical approximation and Markov cbeirs. In [7], the author defined a matrix by using r-normal subdivi of the w,dimensional unit cube, considered it a Markov matrix, and constructed the invariantset and invafiant measure, In fact, the matrix he defined is not Markov matrix generally. So wewill give [7] and amendment in the last pert of this paper. We also construct an invariant set thatis the chain-recurrent set of the map by means of a non-negative matrix which only depends on themap. At hst, we will prove the higher dimension?Banach variation formuls that can simplify thetransition matrix.     

An invariant measure arising in computer simulation of a chaotic dynamical system

Summary Whenf(x)=2x (mod 1) is simulated in a finite discretized space, with random round-off error, the dynamical states can be modeled as belonging to a family of Markov chains. We completely characterize the invariant measure of the discretized dynamics in terms of easily computable stationary measures of the chains.     

随机环境中的马氏链的不变测度与遍历性

本文考虑了一类特殊的随机环境的马氏链。假设随机“Doeblin”条件成立，我们证明了随机环境的马氏链的不变测度存在，且任何初始分布以指数收敛速度到些不变测度。进一步的，存在关于绕积算子遍历的不变测度。最后，我们得到了随机马氏链的强大数定律。     

Symmetric Gibbs measures

We prove that certain Gibbs measures on subshifts of finite type are nonsingular and ergodic for certain countable equivalence relations, including the orbit relation of the adic transformation (the same as equality after a permutation of finitely many coordinates). The relations we consider are defined by cocycles taking values in groups, including some nonabelian ones. This generalizes (half of) the identification of the invariant ergodic probability measures for the Pascal adic transformation as exactly the Bernoulli measures-a version of de Finetti's theorem. Generalizing the other half, we characterize the measures on subshifts of finite type that are invariant under both the adic and the shift as the Gibbs measures whose potential functions depend on only a single coordinate. There are connections with and implications for exchangeability, ratio limit theorems for transient Markov chains, interval splitting procedures, `canonical' Gibbs states, and the triviality of remote sigma-fields finer than the usual tail field.

     

Recurrence and invariant measure of Markov chains in double-infinite random environments

The concepts of π -irreduciblity, recurrence and transience are introduced into the research field of Markov chains in random environments. That a π -irreducible chain must be either recurrent or transient is proved, a criterion is shown for recurrent Markov chains in double-infinite random environments, the existence of invariant measure of π -irreducible chains in double-infinite environments is discussed, and then Orey’s open-questions are partially answered.     

A natural order in dynamical systems based on Conley–Markov matrices

We introduce a natural order to study properties of dynamical systems, especially their invariant sets. The new concept is based on the classical Conley index theory and transition probabilities among neighborhoods of different invariant sets when the dynamical systems are perturbed by white noises. The transition probabilities can be determined by the Fokker–Planck equation and they form a matrix called a Markov matrix. In the limiting case when the random perturbation is reduced to zero, the Markov matrix recovers the information given by the Conley connection matrix. The Markov matrix also produces a natural order from the least to the most stable invariant sets for general dynamical systems. In particular, it gives the order among the local extreme points if the dynamical system is a gradient-like flow of an energy functional. Consequently, the natural order can be used to determine the global minima for gradient-like systems. Some numerical examples are given to illustrate the Markov matrix and its properties.     

Attractors for random dynamical systems

Summary A criterion for existence of global random attractors for RDS is established. Existence of invariant Markov measures supported by the random attractor is proved. For SPDE this yields invariant measures for the associated Markov semigroup. The results are applied to reation diffusion equations with additive white noise and to Navier-Stokes equations with multiplicative and with additive white noise.     

树指标马氏链的等价定义

国内外关于树指标随机过程的研究已经取得了一定的成果.Benjamini和Peres首先给出了树指标马氏链的定义.Berger和叶中行研究了齐次树图上平稳随机场熵率的存在性.杨卫国与刘文研究了树上马氏场的强大数定律与渐近均分性.杨卫国又研究了一般树指标马氏链的强大数定律.为了以后更有效的研究树指标随机过程的一系列相关问题,本文在分析研究前人成果的基础上,给出了树指标马氏链的等价定义,并用数学归纳法证明了其等价性.     

The algebra of reversible Markov chains

For a Markov chain, both the detailed balance condition and the cycle Kolmogorov condition are algebraic binomials. This remark suggests to study reversible Markov chains with the tool of Algebraic Statistics, such as toric statistical models. One of the results of this study is an algebraic parameterization of reversible Markov transitions and their invariant probability.     

Exponential bounds for discrete-time singularly perturbed Markov chains

This paper develops exponential type upper bounds for scaled occupation measures of singularly perturbed Markov chains in discrete time. By considering two-time scale in the Markov chains, asymptotic analysis is carried out. The cases of the fast changing transition probability matrix is irreducible and that are divisible into l ergodic classes are examined first; the upper bounds of a sequence of scaled occupation measures are derived. Then extensions to Markov chains involving transient states and/or nonhomogeneous transition probabilities are dealt with. The results enable us to further our understanding of the underlying Markov chains and related dynamic systems, which is essential for solving many control and optimization problems.     

Drift conditions and invariant measures for Markov chains

We consider the classical Foster–Lyapunov condition for the existence of an invariant measure for a Markov chain when there are no continuity or irreducibility assumptions. Provided a weak uniform countable additivity condition is satisfied, we show that there are a finite number of orthogonal invariant measures under the usual drift criterion, and give conditions under which the invariant measure is unique. The structure of these invariant measures is also identified. These conditions are of particular value for a large class of non-linear time series models.     

Equilibrium states with incomplete supports and periodic trajectories

The relationship between analytic properties of the Artin-Mazur-Ruelle zeta function and the structure of the state of equilibrium states for a topological Markov chain is studied for a class of functions constant on a system of cylinder sets. The convergence of discrete invariant measures to equilibrium states is studied. Special attention is paid to the case in which the uniqueness condition is violated. Dynamic Ruelle-Smale zeta functions are considered, as well as the distribution laws for the number of periodic trajectories of special flows corresponding to topological Markov chains and to positive functions of this class.Translated fromMatematicheskie Zametki, Vol. 59, No. 2, pp. 230–253, February, 1996.The author is grateful to B. M. Gurevich for discussing the paper.This work was partially supported by the International Science Foundation under grant No. M8X000.     

Shift ergodicity for stationary Markov processes

In this paper shift ergodicity and related topics are studied for certain stationary processes. We first present a simple proof of the conclusion that every stationary Markov process is a generalized convex combination of stationary ergodic Markov processes. A direct consequence is that a stationary distribution of a Markov process is extremal if and only if the corresponding stationary Markov process is time ergodic and every stationary distribution is a generalized convex combination of such extremal ones. We then consider space ergodicity for spin flip particle systems. We prove space shift ergodicity and mixing for certain extremal invariant measures for a class of spin systems, in which most of the typical models, such as the Voter Models and the Contact Models, are included. As a consequence of these results we see that for such systems, under each of those extremal invariant measures, the space and time means of an observable coincide, an important phenomenon in statistical physics. Our results provide partial answers to certain interesting problems in spin systems.     

关于随机环境中的马尔可夫过程的简介——纪念李国平院士吴新谋教授诞辰100周年

该文系统地介绍随机环境中的马尔可夫过程. 共4章, 第一章介绍依时的随机环境中的马尔可夫链(MCTRE), 包括MCTRE的存在性及等价描述; 状态分类; 遍历理论及不变测度; p-θ 链的中心极限定理和不变原理. 第二章介绍依时的随机环境中的马尔可夫过程(MPTRE), 包括MPTRE的基本概念; 随机环境中的q -过程存在唯一性; 时齐的q -过程;MPTRE的构造及等价性定理.第三章介绍依时的随机环境中的分枝链(MBCRE), 包括有限维的和无穷维的MBCRE的模型和基本概念; 它们的灭绝概念;两极分化; 增殖率等.第四章介绍依时依空的随机环境中的马尔可夫链(MCSTRE), 包括MCSTRE的基本概念、构造; 依时依空的随机环境中的随机徘徊(RWSTRE)的中心极限定理、不变原理.