随机环境中马氏链的强遍历性

本文研究了随机环境中单链■的强遍历性,得到了单链强遍历的充分条件以及与强遍历性等价的一些形式.利用鞅收敛定理,给出了单链强遍历下尾的结构,最后证明了在环境平稳的条件下,强遍历、平凡尾、弱遍历三者之间的关系,推广了经典马氏链理论中相应的结果.     

A characterization of geometric ergodicity

Summary A homogeneous Markov chain on a countable state space can be classified as ergodic, geometrically ergodic, or strongly ergodic. Ergodicity and strong ergodicity have been characterized using the -coefficient. In this paper the -coefficient is used to characterize geometric ergodicity.     

Generalization bounds of ERM algorithm with V-geometrically Ergodic Markov chains

The previous results describing the generalization ability of Empirical Risk Minimization (ERM) algorithm are usually based on the assumption of independent and identically distributed (i.i.d.) samples. In this paper we go far beyond this classical framework by establishing the first exponential bound on the rate of uniform convergence of the ERM algorithm with V-geometrically ergodic Markov chain samples, as the application of the bound on the rate of uniform convergence, we also obtain the generalization bounds of the ERM algorithm with V-geometrically ergodic Markov chain samples and prove that the ERM algorithm with V-geometrically ergodic Markov chain samples is consistent. The main results obtained in this paper extend the previously known results of i.i.d. observations to the case of V-geometrically ergodic Markov chain samples.     

Technical Note: On Ordinal Comparison of Policies in Markov Reward Processes

An asymptotic exponential convergence rate of ordinal comparison from large deviations theory is well known for selecting the true best solution from the candidate solutions sample means. This note supplements the theories developed by Dai within the framework of ergodic Markov reward processes for -ordinal comparison of policies, establishing an asymptotic exponential convergence rate for the infinite-horizon average criterion.     

Uniform Exponential Ergodicity of Stochastic Dissipative Systems

We study ergodic properties of stochastic dissipative systems with additive noise. We show that the system is uniformly exponentially ergodic provided the growth of nonlinearity at infinity is faster than linear. The abstract result is applied to the stochastic reaction diffusion equation in ^d with d3.     

Generic convergence of infinite products of positive linear operators

We present several results concerning the asymptotic behavior of (random) infinite products of generic sequences of positive linear operators on an ordered Banach space. In addition to a weak ergodic theorem we also obtain convergence to an operator of the formf(·) wheref is a continuous linear functional and is a common fixed point.     

Exact Convergence Rate for the Distributions of GI/M/c/K Queue as K Tends to Infinity

We obtain the exact convergence rate of the stationary distribution ^(K) of the embedded Markov chain in GI/M/c/K queue to the stationary distribution of the embedded Markov chain in GI/M/c queue as K. Similar result for the time-stationary distributions of queue size is also included. These generalize Choi and Kim's results of the case c=1 by nontrivial ways. Our results also strengthen the Simonot's results [5].     

Fuzzy Markov Chains and Decision-Making

Decision-making in an environment of uncertainty and imprecision for real-world problems is a complex task. In this paper it is introduced general finite state fuzzy Markov chains that have a finite convergence to a stationary (may be periodic) solution. The Cesaro average and the -potential for fuzzy Markov chains are defined, then it is shown that the relationship between them corresponds to the Blackwell formula in the classical theory of Markov decision processes. Furthermore, it is pointed out that recurrency does not necessarily imply ergodicity. However, if a fuzzy Markov chain is ergodic, then the rows of its ergodic projection equal the greatest eigen fuzzy set of the transition matrix. Then, the fuzzy Markov chain is shown to be a robust system with respect to small perturbations of the transition matrix, which is not the case for the classical probabilistic Markov chains. Fuzzy Markov decision processes are finally introduced and discussed.     

Strong stationary duality for continuous-time Markov chains. Part I: Theory

LetX(t), 0t<, be an ergodic continuous-time Markov chain with finite or countably infinite state space. We construct astrong stationary dual chainX ^* whose first hitting times yield bounds on the convergence to stationarity forX. The development follows closely the discrete-time theory of Diaconis and Fill.^(2,3) However, for applicability it is important that we formulate our results in terms of infinitesimal rates, and this raises new issues.     

More Randomness of Environment Does Not Always Slow Down a Random Walk

We consider a random walk on in a stationary and ergodic random environment, whose states are called types of the vertices of . We find conditions for which the speed of the random walk is positive. In the case of a Markov chain environment with finitely many states, we give an explicit formula for the speed and for the asymptotic proportion of time spent at vertices of a certain type. Using these results, we compare the speed of random walks on in environments of varying randomness.