随机环境中马氏链的强遍历性

本文研究了随机环境中单链■的强遍历性,得到了单链强遍历的充分条件以及与强遍历性等价的一些形式.利用鞅收敛定理,给出了单链强遍历下尾的结构,最后证明了在环境平稳的条件下,强遍历、平凡尾、弱遍历三者之间的关系,推广了经典马氏链理论中相应的结果.     

A characterization of geometric ergodicity

Summary A homogeneous Markov chain on a countable state space can be classified as ergodic, geometrically ergodic, or strongly ergodic. Ergodicity and strong ergodicity have been characterized using the -coefficient. In this paper the -coefficient is used to characterize geometric ergodicity.     

Generalization bounds of ERM algorithm with V-geometrically Ergodic Markov chains

The previous results describing the generalization ability of Empirical Risk Minimization (ERM) algorithm are usually based on the assumption of independent and identically distributed (i.i.d.) samples. In this paper we go far beyond this classical framework by establishing the first exponential bound on the rate of uniform convergence of the ERM algorithm with V-geometrically ergodic Markov chain samples, as the application of the bound on the rate of uniform convergence, we also obtain the generalization bounds of the ERM algorithm with V-geometrically ergodic Markov chain samples and prove that the ERM algorithm with V-geometrically ergodic Markov chain samples is consistent. The main results obtained in this paper extend the previously known results of i.i.d. observations to the case of V-geometrically ergodic Markov chain samples.     

Uniform Exponential Ergodicity of Stochastic Dissipative Systems

We study ergodic properties of stochastic dissipative systems with additive noise. We show that the system is uniformly exponentially ergodic provided the growth of nonlinearity at infinity is faster than linear. The abstract result is applied to the stochastic reaction diffusion equation in ^d with d3.     

Generic convergence of infinite products of positive linear operators

We present several results concerning the asymptotic behavior of (random) infinite products of generic sequences of positive linear operators on an ordered Banach space. In addition to a weak ergodic theorem we also obtain convergence to an operator of the formf(·) wheref is a continuous linear functional and is a common fixed point.     

Exact Convergence Rate for the Distributions of GI/M/c/K Queue as K Tends to Infinity

We obtain the exact convergence rate of the stationary distribution ^(K) of the embedded Markov chain in GI/M/c/K queue to the stationary distribution of the embedded Markov chain in GI/M/c queue as K. Similar result for the time-stationary distributions of queue size is also included. These generalize Choi and Kim's results of the case c=1 by nontrivial ways. Our results also strengthen the Simonot's results [5].     

Technical Note: On Ordinal Comparison of Policies in Markov Reward Processes

An asymptotic exponential convergence rate of ordinal comparison from large deviations theory is well known for selecting the true best solution from the candidate solutions sample means. This note supplements the theories developed by Dai within the framework of ergodic Markov reward processes for -ordinal comparison of policies, establishing an asymptotic exponential convergence rate for the infinite-horizon average criterion.     

Strong stationary duality for continuous-time Markov chains. Part I: Theory

LetX(t), 0t<, be an ergodic continuous-time Markov chain with finite or countably infinite state space. We construct astrong stationary dual chainX ^* whose first hitting times yield bounds on the convergence to stationarity forX. The development follows closely the discrete-time theory of Diaconis and Fill.^(2,3) However, for applicability it is important that we formulate our results in terms of infinitesimal rates, and this raises new issues.     

Fuzzy Markov Chains and Decision-Making

Decision-making in an environment of uncertainty and imprecision for real-world problems is a complex task. In this paper it is introduced general finite state fuzzy Markov chains that have a finite convergence to a stationary (may be periodic) solution. The Cesaro average and the -potential for fuzzy Markov chains are defined, then it is shown that the relationship between them corresponds to the Blackwell formula in the classical theory of Markov decision processes. Furthermore, it is pointed out that recurrency does not necessarily imply ergodicity. However, if a fuzzy Markov chain is ergodic, then the rows of its ergodic projection equal the greatest eigen fuzzy set of the transition matrix. Then, the fuzzy Markov chain is shown to be a robust system with respect to small perturbations of the transition matrix, which is not the case for the classical probabilistic Markov chains. Fuzzy Markov decision processes are finally introduced and discussed.     

Punktprozesse mit Wechselwirkung

We consider classical, continuous systems of particles in r dimensions described by infinite system equilibrium states which have been defined by Dobrushin [5] and Lanford/Ruelle [24]. For a large class of potentials we prove the theorem of Lee/Yang [43] together with a variational characterizafor these equilibrium states. The main idea stems from Föllmer [9] who showed that in the case of lattice systems, the theorem of Lee/Yang is intimately related to Birkhoff's ergodic theorem and McMillan's theorem (ergodic theorem of information theory). Following this idea we obtain as main results an r-dimensional ergodic theorem for random measures in ^r , limit theorems concerning energy and entropy and an r-dimensional version of Breiman's theorem showing that there is almost sure convergence behind McMillan's theorem.

---

---

Danken möchten wir Klaus Krickeberg, der diese Arbeit durch eine Fülle wertvoller Hinweise und Anregungen gefördert hat.     

More Randomness of Environment Does Not Always Slow Down a Random Walk

We consider a random walk on in a stationary and ergodic random environment, whose states are called types of the vertices of . We find conditions for which the speed of the random walk is positive. In the case of a Markov chain environment with finitely many states, we give an explicit formula for the speed and for the asymptotic proportion of time spent at vertices of a certain type. Using these results, we compare the speed of random walks on in environments of varying randomness.     

Exit probabilities and optimal stochastic control

This paper is concerned with Markov diffusion processes which obey stochastic differential equations depending on a small parameter. The parameter enters as a coefficient in the noise term of the stochastic differential equation. The Ventcel-Freidlin estimates give asymptotic formulas (as0) for such quantities as the probability of exit from a regionD through a given portionN of the boundary D, the mean exit time, and the probability of exit by a given timeT. A new method to obtain such estimates is given, using ideas from stochastic control theory.This research was supported by the Air Force Office of Scientific Research under AF-AFOSR 76-3063, and in part by the National Science Foundation under NSF-MCS 76-37247.     

The Generalized Entropy Ergodic Theorem for Nonhomogeneous Markov Chains

Let \((\xi _n)_{n=0}^\infty \) be a nonhomogeneous Markov chain taking values in a finite state-space \(\mathbf {X}=\{1,2,\ldots ,b\}\). In this paper, we will study the generalized entropy ergodic theorem with almost-everywhere and \(\mathcal {L}_1\) convergence for nonhomogeneous Markov chains; this generalizes the corresponding classical results for Markov chains.     

On asymptotics of the potential of a countable ergodic Markov chain

For a class of functionsf, the convergence in Abel's sense is proved for the potential _no P ⁿ f(i) of a uniform ergodic Markov chain in a countable phase space. Several corollaries are obtained which are useful from the point of view of the possible application to CLT (the central limit theorem) for Markov chains. In particular, we establish the condition equivalent to the boundedness of the second moment for the time of the first return into the state.Translated from Ukrainskii Matematicheskii Zhumal, Vol. 45, No. 2, pp. 265–269, February, 1993.     

Ergodic backward stochastic difference equations

We consider ergodic backward stochastic differential equations in a discrete time setting, where noise is generated by a finite state Markov chain. We show existence and uniqueness of solutions, along with a comparison theorem. To obtain this result, we use a Nummelin splitting argument to obtain ergodicity estimates for a discrete time Markov chain which hold uniformly under suitable perturbations of its transition matrix. We conclude with an application of this theory to a treatment of an ergodic control problem.     

Perfect cocycles through stochastic differential equations

Summary We prove that if is a random dynamical system (cocycle) for whicht(t, )x is a semimartingale, then it is generated by a stochastic differential equation driven by a vector field valued semimartingale with stationary increment (helix), and conversely. This relation is succinctly expressed as semimartingale cocycle=exp(semimartingale helix). To implement it we lift stochastic calculus from the traditional one-sided time to two-sided timeT= and make this consistent with ergodic theory. We also prove a general theorem on the perfection of a crude cocycle, thus solving a problem which was open for more than ten years.This article was processed by the author using the latex style filepljour Im from Springer-Verlag.     

Quasi-likelihood analysis and its applications

The Ibragimov–Khasminskii theory established a scheme that gives asymptotic properties of the likelihood estimators through the convergence of the likelihood ratio random field. This scheme is extending to various nonlinear stochastic processes, combined with a polynomial type large deviation inequality proved for a general locally asymptotically quadratic quasi-likelihood random field. We give an overview of the quasi-likelihood analysis and its applications to ergodic/non-ergodic statistics for stochastic processes.

     

SIA Matrices and Non-Negative Subdivision

We reprove and extend results for uniform convergence of non-negative subdivision using results from infinite products of stochastic matrices as they appear in the study of non-homogeneous finite Markov processes. Geometric convergence of such products to rank one matrices, based on the notions of SIA matrices and of the so-called ergodic coefficient, is discussed. We also point to the properties of the directed graphs of such matrices. In this way the existing convergence results for non-negative subdivision are put into the context of such processes.     

Performance analysis of a slotted-ALOHA protocol on a capture channel with fading

We consider the slotted ALOHA protocol on a channel with a capture effect. There are M < users each with an infinite buffer. If in a slot, i packets are transmitted, then the probability of a successful reception of a packet is q _i. This model contains the CDMA protocols as special cases. We obtain sufficient rate conditions, which are close to necessary for stability of the system, when the arrival streams are stationary ergodic. Under the same rate conditions, for general regenerative arrival streams, we obtain the rates of convergence to stationarity, finiteness of stationary moments and various functional limit theorems. Our arrival streams contain all the traffic models suggested in the recent literature, including the ones which display long range dependence. We also obtain bounds on the stationary moments of waiting times which can be tight under realistic conditions. Finally, we obtain several results on the transient performance of the system, e.g., first time to overflow and the limits of the overflow process. We also extend the above results to the case of a capture channel exhibiting Markov modulated fading. Most of our results and proofs will be shown to hold also for the slotted ALOHA protocol without capture.     

Theoremes ergodiques perturbes

We obtain results on almost sure convergence of ergodic averages along arithmetic subsequences perturbed by independent identically distributed random variables having ap ^th finite moment for somep>0. To prove these results, we use methods based on the harmonic analysis and the theory of Gaussian processes. In fact that will express the stability of Bourgain’s results concerning convergence of ergodic averages for certain arithmetic subsequences.