On strong ergodicity for nonhomogeneous continuous-time Markov chains

Let X(t) be a nonhomogeneous continuous-time Markov chain. Suppose that the intensity matrices of X(t) and some weakly or strongly ergodic Markov chain (t) are close. Some sufficient conditions for weak and strong ergodicity of X(t) are given and estimates of the rate of convergence are proved. Queue-length for a birth and death process in the case of asymptotically proportional intensities is considered as an example.     

Subgeometric ergodicity for continuous-time Markov chains

In this paper, subgeometric ergodicity is investigated for continuous-time Markov chains. Several equivalent conditions, based on the first hitting time or the drift function, are derived as the main theorem. In its corollaries, practical drift criteria are given for ?-ergodicity and computable bounds on subgeometric convergence rates are obtained for stochastically monotone Markov chains. These results are illustrated by examples.     

Uniform stability and weak ergodicity of nonhomogeneous Markov chains defined on ordered Banach spaces with a base

In the present paper, we define an ergodicity coefficient of a positive mapping defined on ordered Banach space with a base , and study its properties. The defined coefficient is a generalization of the well-known the Dobrushin’s ergodicity coefficient. By means of the ergodicity coefficient we provide uniform asymptotical stability conditions for nonhomogeneous discrete Markov chains (NDMC). These results are even new in case of von Neumann algebras. Moreover, we find necessary and sufficient conditions for the weak ergodicity of NDMC. Certain relations between uniform asymptotical stability and weak ergodicity are considered.     

On perturbation bounds for continuous-time Markov chains

We suggest an approach to obtaining general estimates of stability in terms of special “weighted” norms related to total variation. Two important classes of continuous-time Markov chains are considered for which it is possible to obtain exact convergence rate estimates (and hence, guarantee exact stability estimates): birth–death–catastrophes processes, and queueing models with batch arrivals and group services.     

Filtering of continuous-time Markov chains

This paper discusses finite-dimensional optimal filters for partially observed Markov chains. A model for a system containing a finite number of components where each component behaves like an independent finite state continuous-time Markov chain is considered. Using measure change techniques various estimators are derived.     

Inverse problems for ergodicity of Markov chains

For continuous-time Markov chains, we provide criteria for non-ergodicity, non-algebraic ergodicity, non-exponential ergodicity, and non-strong ergodicity. For discrete-time Markov chains, criteria for non-ergodicity, non-algebraic ergodicity, and non-strong ergodicity are given. Our criteria are in terms of the existence of solutions to inequalities involving the Q-matrix (or transition matrix P in time-discrete case) of the chain. Meanwhile, these practical criteria are applied to some examples, including a special class of single birth processes and several multi-dimensional models.     

关于可列非齐次马氏链的广义C-强遍历性

马氏链遍历性理论在生物,数值计算,信息理论,自动控制,近代物理和公用事业中的服务系统等众多领域都有着广泛的应用,马氏链的C-强遍历性是马氏链遍历性理论的重要内容.本文给出了马氏链C-强遍历性的一个推广,首先给出了在可列状态空间取值的非齐次马氏链的广义C-强遍历性和广义一致C-强遍历性的概念,然后研究这两种遍历性成立的充分条件.     

关于非齐次马氏链的一个定理

给出了Csiszar和Krner关于独立随机变量序列的一个定理的一个推广,该定理的推论是关于相对熵的,在统计假设检验及编码理论中起着重要的作用.利用非齐次马氏链的一个强大数定律将这个定理推广到非齐次马氏链上.     

Ergodic degrees for continuous-time Markov chains

This paper studies the existence of the higher orders deviation matrices for continuous time Markov chains by the moments for the hitting times. An estimate of the polynomial convergence rates for the transition matrix to the stationary measure is obtained. Finally, the explicit formulas for birth-death processes are presented.     

Asymptotic inference for continuous-time Markov chains

Summary This paper deals with asymptotic optimal inference in a time-continuous ergodic Markov chain with countable state space, based on observation of the process up to timet. Let the infinitesimal generator depend on an unknown parameter. Under weak assumptions on the parametrization, we show local asymptotic normality for the statistical model ast. As a consequence, limit distributions of sequences of competing estimators for the unknown parameter are more spread out than a specified normal distribution.     

Perturbation analysis for continuous-time Markov chains

We investigate perturbation for continuous-time Markov chains(CTMCs) on a countable state space. Explicit bounds on ?D and D are derived in terms of a drift condition, where ? and D represent the perturbation of the intensity matrices and the deviation matrix, respectively. Moreover, we obtain perturbation bounds on the stationary distributions, which extends the results by Liu(2012) for uniformly bounded CTMCs to general(possibly unbounded) CTMCs. Our arguments are mainly based on the technique of augmented truncations.     

Geometric ergodicity for classes of homogeneous Markov chains

The paper deals with non asymptotic computable bounds for the geometric convergence rate of homogeneous ergodic Markov processes. Some sufficient conditions are stated for simultaneous geometric ergodicity of Markov chain classes. This property is applied to nonparametric estimation in ergodic diffusion processes.     

On the mixing property and the ergodic principle for nonhomogeneous Markov chains

We study different types of limit behavior of infinite dimension discrete time nonhomogeneous Markov chains. We show that the geometric structure of the set of those Markov chains which have asymptotically stationary density depends on the considered topologies. We generalize and correct some results from Ganikhodjaev et al. (2006) [3].     

Risk-sensitive stopping problems for continuous-time Markov chains

In this paper we consider stopping problems for continuous-time Markov chains under a general risk-sensitive optimization criterion for problems with finite and infinite time horizon. More precisely our aim is to maximize the certainty equivalent of the stopping reward minus cost over the time horizon. We derive optimality equations for the value functions and prove the existence of optimal stopping times. The exponential utility is treated as a special case. In contrast to risk-neutral stopping problems it may be optimal to stop between jumps of the Markov chain. We briefly discuss the influence of the risk sensitivity on the optimal stopping time and consider a special house selling problem as an example.     

Geometric ergodicity and the spectral gap of non-reversible Markov chains

We argue that the spectral theory of non-reversible Markov chains may often be more effectively cast within the framework of the naturally associated weighted-L _∞ space ${L_\infty^V}$ , instead of the usual Hilbert space L ₂?=?L ₂(π), where π is the invariant measure of the chain. This observation is, in part, based on the following results. A discrete-time Markov chain with values in a general state space is geometrically ergodic if and only if its transition kernel admits a spectral gap in ${L_\infty^V}$ . If the chain is reversible, the same equivalence holds with L ₂ in place of ${L_\infty^V}$ . In the absence of reversibility it fails: There are (necessarily non-reversible, geometrically ergodic) chains that admit a spectral gap in ${L_\infty^V}$ but not in L ₂. Moreover, if a chain admits a spectral gap in L ₂, then for any ${h\in L_2}$ there exists a Lyapunov function ${V_h\in L_1}$ such that V _h dominates h and the chain admits a spectral gap in ${L_\infty^{V_h}}$ . The relationship between the size of the spectral gap in ${L_\infty^V}$ or L ₂, and the rate at which the chain converges to equilibrium is also briefly discussed.     

Geometric ergodicity in a class of denumerable Markov chains

We study the question of geometric ergodicity in a class of Markov chains on the state space of non-negative integers for which, apart from a finite number of boundary rows and columns, the elements pjk of the one-step transition matrix are of the form c _k-j where {c _k} is a probability distribution on the set of integers. Such a process may be described as a general random walk on the non-negative integers with boundary conditions affecting transition probabilities into and out of a finite set of boundary states. The imbedded Markov chains of several non-Markovian queueing processes are special cases of this form. It is shown that there is an intimate connection between geometric ergodicity and geometric bounds on one of the tails of the distribution {c _k}.This research was supported by the U.S. office of Naval Research Contract No. Nonr-855(09), and carried out while the author was a visitor in the Statistics department, University of North Carolina, Chapel Hill.     

Remarks on ergodicity of stationary irreducible transient Markov chains

Summary Consider a stationary process {X _n(), – < n < . If the measure of the process is finite (the measure of the whole sample space finite), it is well known that ergodicity of the process {X _n(), - < n < and of each of the subprocesses {X _n(), 0 n < , {X _n(), – < n 0 are equivalent (see [3]). We shall show that this is generally not true for stationary processes with a sigma-finite measure, specifically for stationary irreducible transient Markov chains. An example of a stationary irreducible transient Markov chain {X _n(), - < n <} with {itX_n(), 0 n < < ergodic but {X _n(), < n 0 nonergodic is given. That this can be the case has already been implicitly indicated in the literature [4]. Another example of a stationary irreducible transient Markov chain with both {X _n(), 0 n < and {itX _n(),-< < n 0} ergodic but {X _n(), - < n < nonergodic is presented. In fact, it is shown that all stationary irreducible transient Markov chains {X _n(), - < n < < are nonergodic.This research was supported in part by the Office of Naval Research.John Simon Guggenheim Memorial Fellow.