Chutes and Ladders in Markov Chains

We investigate how the stationary distribution of a Markov chain changes when transitions from a single state are modified. In particular, adding a single directed edge to nearest neighbor random walk on a finite discrete torus in dimensions one, two, or three changes the stationary distribution linearly, logarithmically, or only locally. Related results are derived for birth and death chains approximating Bessel diffusions and for random walk on the Sierpinski gasket.     

On Multivariate Markov Chains for Common and Non-Common Objects in Multiple Networks

Node importance or centrality evaluation is an important methodology for network analysis. In this paper, we are interested in the study of objects appearing in several networks. Such common objects are important in network-network interactions via object-object interactions. The main contribution of this paper is to model multiple networks where there are some common objects in a multivariate Markov chain framework, and to develop a method for solving common and non-common objects' stationary probability distributions in the networks. The stationary probability distributions can be used to evaluate the importance of common and non-common objects via network-network interactions. Our experimental results based on examples of co-authorship of researchers in different conferences and paper citations in different categories have shown that the proposed model can provide useful information for researcher-researcher interactions in networks of different conferences and for paper-paper interactions in networks of different categories.     

Factorization of Markov Chains

Existence of following factorization is proved:

Self-Interacting Markov Chains

Abstract

In this article we study a class of self-interacting Markov chain models. We propose a novel theoretical basis based on measure-valued processes and semigroup techniques to analyze its asymptotic behavior as the time parameter tends to infinity. We exhibit different types of decays to equilibrium, depending on the level of interaction. We illustrate these results in a variety of examples, including Gaussian or Poisson self-interacting models. We analyze the long-time behavior of a new class of evolutionary self-interacting chain models. These genetic type algorithms can also be regarded as reinforced stochastic explorations of an environment with obstacles related to a potential function.     

肖争艳等： 绕积马氏链的状态分类

该文给出了绕积马氏链的特征数和状态的定义, 利用一般马氏链的理论讨论了随机环 境中的马氏链的各种状态的特征以及各类状态之间的联系, 还给出了在联合空间不可分解且 正则本质的条件下, 状态正则本质的充要条件. 最后举例说明了经典马氏链和随机环境中马氏链的状态的区别.     

Geometric Convergence Rates for Time-Sampled Markov Chains

We consider time-sampled Markov chain kernels, of the form P = _n n P ⁿ . We prove bounds on the total variation distance to stationarity of such chains. We are motivated by the analysis of near-periodic MCMC algorithms.     

马氏环境中马氏链的Poisson极限律

本文研究了马氏环境中马氏链,证明了该过程于小柱集上的回返次数是渐近地服从Poisson分布的,同时还给出了该过程是(?)-混合的一个充分条件以及过程回返于小柱集之概率的一个指数估计式.     

Probabilities for Queueing Systems with Embedded Markov Chains

Abstract

Transition probabilities of embedded Markov chain for single-server queues are considered when the distribution of the inter-arrival time or that of the service time is specified. A comprehensive collection of formulas is derived for the transition probabilities, covering some seventeen flexible families. The corresponding estimation procedures are also derived by the method of moments. It is expected that this work could serve as a useful reference for the modeling of queuing systems with embedded Markov chains.     

Block Two-stage Methods for Singular Systems and Markov Chains

The use of block two-stage methods for the iterative solution of consistent singular linear systems is studied. In these methods, suitable for parallel computations, different blocks, i.e., smaller linear systems, can be solved concurrently by different processors. Each of these smaller systems are solved by an (inner) iterative method. Hypotheses are provided for the convergence of non-stationary methods, i.e., when the number of inner iterations may vary from block to block and from one outer iteration to another. It is shown that the iteration matrix corresponding to one step of the block method is convergent, i.e., that its powers converge to a limit matrix. A theorem on the convergence of the infinite product of matrices with the same eigenspace corresponding to the eigenvalue 1 is proved, and later used as a tool in the convergence analysis of the block method. The methods studied can be used to solve any consistent singular system, including discretizations of certain differential equations. They can also be used to find stationary probability distribution of Markov chains. This last application is considered in detail.     

绕积马氏链的几个结果

本文利用一般马氏链的理论讨论了随机环境中的马氏链的各种状态的特征及遍历性,并用两种方式将状态空间进行严格的分类.     

马氏环境中的马氏链与马氏双链

本文研究了马氏环境中的马氏链,利用马氏双链的性质,得到了马氏环境中的马氏链回返于小柱集上的概率的若干估计式.     

关于马氏环境中马氏链的几点注记

讨论了马氏环境中马氏链与马氏双链间的关系,通过两个例子,纠正了有关文献的一些错误结论。     

树指标集马氏链的常返性

用鞅方法给出了在不限定根顶点O的条件下,树指标集马氏链常返性的充分条件,部分解决了Benjamin.I提出的问题.     

树指标马氏链的等价定义

国内外关于树指标随机过程的研究已经取得了一定的成果.Benjamini和Peres首先给出了树指标马氏链的定义.Berger和叶中行研究了齐次树图上平稳随机场熵率的存在性.杨卫国与刘文研究了树上马氏场的强大数定律与渐近均分性.杨卫国又研究了一般树指标马氏链的强大数定律.为了以后更有效的研究树指标随机过程的一系列相关问题,本文在分析研究前人成果的基础上,给出了树指标马氏链的等价定义,并用数学归纳法证明了其等价性.     

随机环境中马氏链与马氏双链间的相互关系

比较圆满地解决了单无限和双无限环境及其对应的双链和原过程四者间的相互关系．特别地,澄清了一些误解,纠正了其中的错误结论,为进一步深入研究随机环境中马氏链提供了非常明晰的基本概念．     

二叉树上分支马氏链的等价性质

本文给出了二叉树上分支马氏链定义的离散形式,然后研究了它的两个等价性质.最后,我们指出在二叉树情况下,树指标马氏链就是一类特殊的分支马氏链.     

Approximate uniformization for continuous-time Markov chains with an application to performability analysis

An approximate version of the standard uniformization technique is introduced for application to continuous-time Markov chains with unbounded jump rates. This technique is shown to be asymptotically exact and an error bound for the order of its accuracy is provided. An illustrative queueing application is included.     

Continuous-Time Controlled Markov Chains with Discounted Rewards

This paper studies denumerable state continuous-time controlled Markov chains with the discounted reward criterion and a Borel action space. The reward and transition rates are unbounded, and the reward rates are allowed to take positive or negative values. First, we present new conditions for a nonhomogeneous Q(t)-process to be regular. Then, using these conditions, we give a new set of mild hypotheses that ensure the existence of -optimal (0) stationary policies. We also present a martingale characterization of an optimal stationary policy. Our results are illustrated with controlled birth and death processes.     

随机环境中马氏链的Chacon-Ornstein定理

利用随机环境中马氏链的Hofp极大遍历引理和Brunel极大遍历引理,给出了随机环境中马氏链的Chacon-Ornstein定理和Chacon认证定理.