关于连续状态非齐次马氏链的绝对平均强遍历性

引进连续状态非齐次马氏链绝对平均强遍历的概念,研究连续状态非齐次马氏链满足这种强遍历的一个充分条件,并给出绝对平均强遍历性在马氏决策过程中的应用.     

马氏环境中马氏链的一个概率不等式及其应用

本文研究了马氏环境中马氏链构成的随机变量之和的概率不等式问题.利用了结尾的方法,获得了马氏环境中马氏链构成的随机变量之和的尾部概率不等式,作为结果的应用,给出了将过程限制在(S,S∩F,PS)上的强大数定律.文中提出的方法和结果对研究独立的随机变量之和的大样本性质是十分有用的.     

依状态独立的随机环境中的马氏链

本主要讨论了依状态独立的随机环境中的马氏链，并严格地证明了依状态独立的随机环境中的马氏链，如果在环境不退化的一般情形下，不是时齐马氏链，而环境退化必然是马氏链这个结论。     

肖争艳等： 绕积马氏链的状态分类

该文给出了绕积马氏链的特征数和状态的定义, 利用一般马氏链的理论讨论了随机环 境中的马氏链的各种状态的特征以及各类状态之间的联系, 还给出了在联合空间不可分解且 正则本质的条件下, 状态正则本质的充要条件. 最后举例说明了经典马氏链和随机环境中马氏链的状态的区别.     

马氏环境中的马氏链与马氏双链

本文研究了马氏环境中的马氏链,利用马氏双链的性质,得到了马氏环境中的马氏链回返于小柱集上的概率的若干估计式.     

状态可数的马氏环境中马氏链函数的强大数定律

讨论了马氏双链与随机环境中马氏链的关系．在此基础上，研究了具有离散参量的马氏环境中马氏链函数的强大数定律，并且给出了直接加于链和过程样本函数上的充分条件．     

马氏环境中马氏链的Poisson极限律

本文研究了马氏环境中马氏链,证明了该过程于小柱集上的回返次数是渐近地服从Poisson分布的,同时还给出了该过程是(?)-混合的一个充分条件以及过程回返于小柱集之概率的一个指数估计式.     

关于马氏链在常返态下定理的一个新的证明

本文通过马氏链在常返状态下的已知定理,利用常返性质,给出该定理的一个新的证明.     

关于吸收马氏链的注记

设Ｐ０＝（Ｐｉｊ）是一个状态空间为Ｅ＝（０，１，２，．．．）的吸收马氏链，其中“０”是唯一的吸收状态，其它状态构成一个非本质类，定义一个新链Ｐ１＝（Ｐｉｊ／１－ｐｉ０）ｉ，ｊ∈Ｅ１，Ｅ１＝（１，２，．．．）。讨论了Ｐ１的状态性质（常返，正常返）与Ｐ０的吸收性质之间的关系。     

����״̬�����������Եij���֤��

In this paper, we introduce the definitions of geometric strongly ergodic, strongly ergodic and weakly ergodic for continuous-state Markov chains, then we give a primary proof of equivalence of the ergodicities for continuous-state Markov chains.     

Bounds on Lifting Continuous-State Markov Chains to Speed Up Mixing

It is often possible to speed up the mixing of a Markov chain \(\{ X_{t} \}_{t \in \mathbb {N}}\) on a state space \(\Omega \) by lifting, that is, running a more efficient Markov chain \(\{ \widehat{X}_{t} \}_{t \in \mathbb {N}}\) on a larger state space \(\hat{\Omega } \supset \Omega \) that projects to \(\{ X_{t} \}_{t \in \mathbb {N}}\) in a certain sense. Chen et al. (Proceedings of the 31st annual ACM symposium on theory of computing. ACM, 1999) prove that for Markov chains on finite state spaces, the mixing time of any lift of a Markov chain is at least the square root of the mixing time of the original chain, up to a factor that depends on the stationary measure of \(\{X_t\}_{t \in \mathbb {N}}\). Unfortunately, this extra factor makes the bound in Chen et al. (1999) very loose for Markov chains on large state spaces and useless for Markov chains on continuous state spaces. In this paper, we develop an extension of the evolving set method that allows us to refine this extra factor and find bounds for Markov chains on continuous state spaces that are analogous to the bounds in Chen et al. (1999). These bounds also allow us to improve on the bounds in Chen et al. (1999) for some chains on finite state spaces.     

General Bernstein-Like Inequality for Additive Functionals of Markov Chains

Journal of Theoretical Probability - Using the renewal approach, we prove Bernstein-like inequalities for additive functionals of geometrically ergodic Markov chains, thus obtaining counterparts of...     

Expectations for Nonreversible Markov Chains

Bounds are given for an irreducible Markov chain on the probability that the time average of a functional on the state space exceeds its stationary expectation, without assuming reversibility. The bounds are in terms of the singular values of the discrete generator.     

Measure-Valued Differentiation for Markov Chains

This paper addresses the problem of sensitivity analysis for finite-horizon performance measures of general Markov chains. We derive closed-form expressions and associated unbiased gradient estimators for the derivatives of finite products of Markov kernels by measure-valued differentiation (MVD). In the MVD setting, the derivatives of Markov kernels, called -derivatives, are defined with respect to a class of performance functions such that, for any performance measure , the derivative of the integral of g with respect to the one-step transition probability of the Markov chain exists. The MVD approach (i) yields results that can be applied to performance functions out of a predefined class, (ii) allows for a product rule of differentiation, that is, analyzing the derivative of the transition kernel immediately yields finite-horizon results, (iii) provides an operator language approach to the differentiation of Markov chains and (iv) clearly identifies the trade-off between the generality of the performance classes that can be analyzed and the generality of the classes of measures (Markov kernels). The -derivative of a measure can be interpreted in terms of various (unbiased) gradient estimators and the product rule for -differentiation yields a product-rule for various gradient estimators. Part of this work was done while the first author was with EURANDOM, Eindhoven, Netherlands, where he was supported by Deutsche Forschungsgemeinschaft under Grant He3139/1-1. The work of the second author was partially supported by NSERC and FCAR grants of the Government of Canada and Québec.     

马氏链平稳分布存在与唯一性的简洁证明与计算

本文考虑可数状态离散时间齐次马氏链平稳分布的存在与唯一性.放弃以往大多数文献中要求马氏链是不可约,正常返且非周期(即遍历)的条件,本文仅需要马氏链是不可约和正常返的(但可能是周期的,因而可能是非遍历的).在此较弱的条件下,本文不仅给出了平稳分布存在与唯一性的简洁证明,而且还给出了平稳分布的计算方法.     

Segregating Markov Chains

Dealing with finite Markov chains in discrete time, the focus often lies on convergence behavior and one tries to make different copies of the chain meet as fast as possible and then stick together. There are, however, discrete finite (reducible) Markov chains, for which two copies started in different states can be coupled to meet almost surely in finite time, yet their distributions keep a total variation distance bounded away from 0, even in the limit as time tends to infinity. We show that the supremum of total variation distance kept in this context is \(\tfrac{1}{2}\).     

Self-Interacting Markov Chains

Abstract

In this article we study a class of self-interacting Markov chain models. We propose a novel theoretical basis based on measure-valued processes and semigroup techniques to analyze its asymptotic behavior as the time parameter tends to infinity. We exhibit different types of decays to equilibrium, depending on the level of interaction. We illustrate these results in a variety of examples, including Gaussian or Poisson self-interacting models. We analyze the long-time behavior of a new class of evolutionary self-interacting chain models. These genetic type algorithms can also be regarded as reinforced stochastic explorations of an environment with obstacles related to a potential function.     

Generalized Dobrushin ergodicity coefficient and uniform ergodicities of Markov operators

Positivity - In this paper the stability and the perturbation bounds of Markov operators acting on abstract state spaces are investigated. Here, an abstract state space is an ordered Banach space...