马氏环境中马氏链的一类强极限定理

利用分析方法研究了马氏环境中马氏链的若干强极限定理.得到了关于此种链四元函数的一个强极限定理.作为推论,得到了马氏环境中马氏链相对熵密度的几个极限性质,将Shannon定理推广到了马氏环境中马氏链的情况.     

马氏环境中可数马氏链的Poisson极限率

研究了马氏环境中的可数马氏链,主要证明了过程于小柱集上的回返次数是渐近地服从Poisson分布。为此,引入熵函数h,首先给出了马氏环境中马氏链的Shannon-Mc Millan-Breiman定理,还给出了一个非马氏过程Posson逼近的例子。当环境过程退化为一常数序列时,便得到可数马氏链的Poisson极限定理。这是有限马氏链Pitskel相应结果的拓广。     

从p-m链到随机环境中的马氏链

第一节引进了p-m链的概念,并用之构造了与它相应的随机环境中的马氏链和绕积马氏链.第二节引进了一系列与随机环境中的马氏链相关的概率特性函数,并得到了这些函数之间的一系列关系.这些结果是经典马氏链的相应结果的一般化,它们在随机环境中的马氏链的极限理论的研究中是很有用的.     

状态可数的马氏环境中马氏链函数的强大数定律

讨论了马氏双链与随机环境中马氏链的关系．在此基础上，研究了具有离散参量的马氏环境中马氏链函数的强大数定律，并且给出了直接加于链和过程样本函数上的充分条件．     

马氏环境中马氏链的中心极限定理

讨论了具有离散参数的马氏环境中马氏链的中心极限定理, 并给出了加在链和过程样本函数上的充分条件\bd 同时深入研究了$R_{\theta}$\,-链, 得到马氏环境中马氏链的中心极限定理成立的三个充分条件.     

依状态独立的随机环境中的马氏链

本主要讨论了依状态独立的随机环境中的马氏链，并严格地证明了依状态独立的随机环境中的马氏链，如果在环境不退化的一般情形下，不是时齐马氏链，而环境退化必然是马氏链这个结论。     

马氏环境中马氏链泛函的中心极限定理

讨论了具有离散参数的马氏环境中马氏链的性质,建立了马氏环境中马氏链泛函的中心极限定理.同时给出了加在链和过程样本函数上的充分条件.     

马氏环境中马氏链的强大数定律

讨论了具有离散参数的马氏环境中马氏链的强大数定律，并给出了加在链和过程样本函数上的充分条件．同时深入研究了Rθ-链，得到马氏环境中马氏链强大数定律成立的充分条件．     

从p—m链到随机环境中的马氏链

第一节引进了p一m链的概念,并用之构造了与它相应的随机环境中的马氏链和绕积马氏链、第二节引进了一系列与随机环境中的马氏链相关的概率特性函数,并得到了这些函数之间的一系列关系.这些结果是经典马氏链的相应结果的一般化,它们在随机环境中的马氏链的极限理论的研究中是很有用的。     

马氏环境中马氏链函数加权和的强收敛性

给出了状态有限的单无限马氏环境中马氏链泛函加权和的强收敛性,得到了状态有限的单无限马氏环境中马氏链泛函加权和的强收敛性成立的一系列充分条件.     

马氏环境中马氏链的Poisson极限律

本文研究了马氏环境中马氏链,证明了该过程于小柱集上的回返次数是渐近地服从Poisson分布的,同时还给出了该过程是(?)-混合的一个充分条件以及过程回返于小柱集之概率的一个指数估计式.     

一类随机环境下马尔科夫链状态的分类

本文构造出具有两状态的一类平稳遍历的环境列下马尔科夫链的具体模型 ,并对该环境下马尔科夫链的相空间进行分解 .     

Fuzzy Markov Chains and Decision-Making

Decision-making in an environment of uncertainty and imprecision for real-world problems is a complex task. In this paper it is introduced general finite state fuzzy Markov chains that have a finite convergence to a stationary (may be periodic) solution. The Cesaro average and the -potential for fuzzy Markov chains are defined, then it is shown that the relationship between them corresponds to the Blackwell formula in the classical theory of Markov decision processes. Furthermore, it is pointed out that recurrency does not necessarily imply ergodicity. However, if a fuzzy Markov chain is ergodic, then the rows of its ergodic projection equal the greatest eigen fuzzy set of the transition matrix. Then, the fuzzy Markov chain is shown to be a robust system with respect to small perturbations of the transition matrix, which is not the case for the classical probabilistic Markov chains. Fuzzy Markov decision processes are finally introduced and discussed.     

Separable solutions for Markov processes in random environments

In this paper we address the problem of efficiently deriving the steady-state distribution for a continuous time Markov chain (CTMC) S evolving in a random environment E. The process underlying E is also a CTMC. S is called Markov modulated process. Markov modulated processes have been widely studied in literature since they are applicable when an environment influences the behaviour of a system. For instance, this is the case of a wireless link, whose quality may depend on the state of some random factors such as the intensity of the noise in the environment. In this paper we study the class of Markov modulated processes which exhibits separable, product-form stationary distribution. We show that several models that have been proposed in literature can be studied applying the Extended Reversed Compound Agent Theorem (ERCAT), and also new product-forms are derived. We also address the problem of the necessity of ERCAT for product-forms and show a meaningful example of product-form not derivable via ERCAT.     

Markov modulated Bernoulli process

We consider a Bernoulli process where the success probability changes with respect to a Markov chain. Such a model represents an interesting application of stochastic processes where the parameters are not constants; rather, they are stochastic processes themselves due to their dependence on a randomly changing environment. The model operates in a random environment depicted by a Markov chain so that the probability of success at each trial depends on the state of the environment. We will concentrate, in particular, on applications in reliability theory to motivate our model. The analysis will focus on transient as well as long-term behaviour of various processes involved.     

Self-Interacting Markov Chains

Abstract

In this article we study a class of self-interacting Markov chain models. We propose a novel theoretical basis based on measure-valued processes and semigroup techniques to analyze its asymptotic behavior as the time parameter tends to infinity. We exhibit different types of decays to equilibrium, depending on the level of interaction. We illustrate these results in a variety of examples, including Gaussian or Poisson self-interacting models. We analyze the long-time behavior of a new class of evolutionary self-interacting chain models. These genetic type algorithms can also be regarded as reinforced stochastic explorations of an environment with obstacles related to a potential function.     

Stopping times for quantum Markov chains

In the paper we introduce stopping times for quantum Markov states. We study algebras and maps corresponding to stopping times, give a condition of strong Markov property and give classification of projections for the property of accessibility. Our main result is a new recurrence criterium in terms of stopping times (Theorem 1 and Corollary 2). As an application of the criterium we study how, in Section 6, the quantum Markov chain associated with the one-dimensional Heisenberg (usually non-Markovian) process, obtained from this quantum Markov chain by restriction to a diagonal subalgebra, is such that all its states are recurrent. We were not able to obtain this result from the known recurrence criteria of classical probability.Supported by GNAFA-CNR, Bando n. 211.01.25.     

关于随机环境中的马尔可夫过程的简介——纪念李国平院士吴新谋教授诞辰100周年

该文系统地介绍随机环境中的马尔可夫过程. 共4章, 第一章介绍依时的随机环境中的马尔可夫链(MCTRE), 包括MCTRE的存在性及等价描述; 状态分类; 遍历理论及不变测度; p-θ 链的中心极限定理和不变原理. 第二章介绍依时的随机环境中的马尔可夫过程(MPTRE), 包括MPTRE的基本概念; 随机环境中的q -过程存在唯一性; 时齐的q -过程;MPTRE的构造及等价性定理.第三章介绍依时的随机环境中的分枝链(MBCRE), 包括有限维的和无穷维的MBCRE的模型和基本概念; 它们的灭绝概念;两极分化; 增殖率等.第四章介绍依时依空的随机环境中的马尔可夫链(MCSTRE), 包括MCSTRE的基本概念、构造; 依时依空的随机环境中的随机徘徊(RWSTRE)的中心极限定理、不变原理.