Classical limit theorems for measure-valued Markov processes

Laws of large numbers, central limit theorems, and laws of the iterated logarithm are obtained for discrete and continuous time Markov processes whose state space is a set of measures. These results apply to each measure-valued stochastic process itself and not simply to its real-valued functionals.     

On an extremal property of Markov chains and sufficiency of Markov strategies in Markov decision processes with the Dubins-Savage criterion

An inequality regarding the minimum ofP(lim inf(X _n D _n )) is proved for a class of random sequences. This result is related to the problem of sufficiency of Markov strategies for Markov decision processes with the Dubins-Savage criterion, the asymptotical behaviour of nonhomogeneous Markov chains, and some other problems.     

Waiting time distributions of runs in higher order Markov chains

We consider a {0,1}-valuedm-th order stationary Markov chain. We study the occurrences of runs where two 1’s are separated byat most/exactly/at least k 0’s under the overlapping enumeration scheme wherek≥0 and occurrences of scans (at leastk ₁ successes in a window of length at mostk, 1≤k ₁≤k) under both non-overlapping and overlapping enumeration schemes. We derive the generating function of first two types of runs. Under the conditions, (1) strong tendency towards success and (2) strong tendency towards reversing the state, we establish the convergence of waiting times of ther-th occurrence of runs and scans to Poisson type distributions. We establish the central limit theorem and law of the iterated logarithm for the number of runs and scans up to timen.     

Sooner and later waiting time problems in a two-state Markov chain

LetX ₁,X ₂,... be a time-homogeneous {0, 1}-valued Markov chain. LetF ₀ be the event thatl runs of 0 of lengthr occur and letF ₁ be the event thatm runs of 1 of lengthk occur in the sequenceX ₁,X ₂, ... We obtained the recurrence relations of the probability generating functions of the distributions of the waiting time for the sooner and later occurring events betweenF ₀ andF ₁ by the non-overlapping way of counting and overlapping way of counting. We also obtained the recurrence relations of the probability generating functions of the distributions of the sooner and later waiting time by the non-overlapping way of counting of 0-runs of lengthr or more and 1-runs of lengthk or more.