Exponential convergence in ‐Wasserstein distance for diffusion processes without uniformly dissipative drift

By adopting the coupling by reflection and choosing an auxiliary function which is convex near infinity, we establish the exponential convergence of diffusion semigroups with respect to the standard ‐Wasserstein distance for all . In particular, we show that for the Itô stochastic differential equation if the drift term b is such that for any , holds with some positive constants K₁, K₂ and , then there is a constant such that for all , and , where is a positive constant. This improves the main result in 14 where the exponential convergence is only proved for the L¹‐Wasserstein distance.     

Bounds on regeneration times and convergence rates for Markov chains

In many applications of Markov chains, and especially in Markov chain Monte Carlo algorithms, the rate of convergence of the chain is of critical importance. Most techniques to establish such rates require bounds on the distribution of the random regeneration time T that can be constructed, via splitting techniques, at times of return to a “small set” C satisfying a minorisation condition P(x,·)(·), xC. Typically, however, it is much easier to get bounds on the time τ_C of return to the small set itself, usually based on a geometric drift function , where . We develop a new relationship between T and τ_C, and this gives a bound on the tail of T, based on ,λ and b, which is a strict improvement on existing results. When evaluating rates of convergence we see that our bound usually gives considerable numerical improvement on previous expressions.     

On the convergence of sequences of stationary jump Markov processes

This paper presents two main results: first, a Liapunov type criterion for the existence of a stationary probability distribution for a jump Markov process; second, a Liapunov type criterion for existence and tightness of stationary probability distributions for a sequence of jump Markov processes. If the corresponding semigroups T_N(t) converge, under suitable hypotheses on the limit semigroup, this last result yields the weak convergence of the sequence of stationary processes (T_N(t), π_N) to the stationary limit one.     

Markov processes with darning and their approximations

In this paper, we study darning of general symmetric Markov processes by shorting some parts of the state space into singletons. A natural way to construct such processes is via Dirichlet forms restricted to the function spaces whose members take constant values on these collapsing parts. They include as a special case Brownian motion with darning, which has been studied in details in Chen (2012), Chen and Fukushima (2012) and Chen et al. (2016). When the initial processes have discontinuous sample paths, the processes constructed in this paper are the genuine extensions of those studied in Chen and Fukushima (2012). We further show that, up to a time change, these Markov processes with darning can be approximated in the sense of finite-dimensional distributions by introducing additional jumps with large intensity among these compact sets to be collapsed into singletons. For diffusion processes, it is also possible to get, up to a time change, diffusions with darning by increasing the conductance on these compact sets to infinity. To accomplish these, we give a version of the semigroup characterization of Mosco convergence to closed symmetric forms whose domain of definition may not be dense in the $L^2$-space. The latter is of independent interest and potentially useful to study convergence of Markov processes having different state spaces. Indeed, we show in Section 5 of this paper that Brownian motion in a plane with a very thin flag pole can be approximated by Brownian motion in the plane with a vertical cylinder whose horizontal motion on the cylinder is a circular Brownian motion moving at fast speed.     

Coupling, convergence rates of Markov processes and weak Poincaré inequalities

Some analytic and probabilistic properties of the weak Poincaré inequality are obtained. In particular, for strong Feller Markov processes the existence of this inequality is equivalent to each of the following: (i)the Liouville property (or the irreducibility); (ii) the existence of successful couplings (or shift-couplings); (iii)the convergence of the Markov process in total variation norm; (iv) the triviality of the tail (or the invariant)σ-field; (v) the convergence of the density. Estimates of the convergence rate in total variation norm of Markov processes are obtained using the weak Poincaré inequality.     

Markov chain approximations to filtering equations for reflecting diffusion processes

Herein, we consider direct Markov chain approximations to the Duncan–Mortensen–Zakai equations for nonlinear filtering problems on regular, bounded domains. For clarity of presentation, we restrict our attention to reflecting diffusion signals with symmetrizable generators. Our Markov chains are constructed by employing a wide band observation noise approximation, dividing the signal state space into cells, and utilizing an empirical measure process estimation. The upshot of our approximation is an efficient, effective algorithm for implementing such filtering problems. We prove that our approximations converge to the desired conditional distribution of the signal given the observation. Moreover, we use simulations to compare computational efficiency of this new method to the previously developed branching particle filter and interacting particle filter methods. This Markov chain method is demonstrated to outperform the two-particle filter methods on our simulated test problem, which is motivated by the fish farming industry.     

Exponential rate of convergence for some Markov operators

The exponential rate of convergence for Markov operators is established. The operators correspond to continuous iterated function systems which are a very useful tool in some cell cycle models.     

Error bounds for augmented truncation approximations of continuous-time Markov chains

This paper considers the augmented truncation approximation of the generator of an ergodic continuous-time Markov chain with a countably infinite state space. The main purpose of this paper is to present bounds for the absolute difference between the stationary distributions of the original generator and its augmented truncation. As examples, we apply the bounds to an $M∕M∕s$ retrial queue and an upper Hessenberg Markov chain.     

Weak convergence of Markov-modulated random sequences

This work is concerned with weak convergence of non-Markov random processes modulated by a Markov chain. The motivation of our study stems from a wide variety of applications in actuarial science, communication networks, production planning, manufacturing and financial engineering. Owing to various modelling considerations, the modulating Markov chain often has a large state space. Aiming at reduction of computational complexity, a two-time-scale formulation is used. Under this setup, the Markov chain belongs to the class of nearly completely decomposable class, where the state space is split into several subspaces. Within each subspace, the transitions of the Markov chain varies rapidly, and among different subspaces, the Markov chain moves relatively infrequently. Aggregating all the states of the Markov chain in each subspace to a single super state leads to a new process. It is shown that under such aggregation schemes, a suitably scaled random sequence converges to a switching diffusion process.     

On the speed of convergence in the central limit theorem of log-likelihood ratio processes

Let , the parameter space, be an open subset ofR ^k ,k1. For each , let the r.v.'sX _n ,n=1, 2,... be defined on the probability space (X, P ) and take values in (S,S,L) whereS is a Borel subset of a Euclidean space andL is the -field of Borel subsets ofS. ForhR _k and a sequence of p.d. normalizing matrices _n = _n ^{k × k} (₀ set _n ^* = ^* = ₀ + _n h, where ₀ is the true value of , such that ^*, . Let _n (^*, _*)( be the log-likelihood ratio of the probability measure with respect to the probability measure , whereP _n is the restriction ofP over _n = (X ₁,X ₂,...,X _n . In this paper we, under a very general dependence setup obtain a rate of convergence of the normalized log-likelihood ratio statistic to Standard Normal Variable. Two examples are taken into account.     

The convergence of Cesaro averages for certain nonstationary Markov chains

If P is a stochastic matrix corresponding to a stationary, irreducible, positive persistent Markov chain of period d>1, the powers Pⁿ will not converge as n → ∞. However, the subsequences P^nd+k for k=0,1,...d-1, and hence Cesaro averages Σⁿ_k-1 P^k/n, will converge. In this paper we determine classes of nonstationary Markov chains for which the analogous subsequences and/or Cesaro averages converge and consider the rates of convergence. The results obtained are then applied to the analysis of expected average cost.     

Coupling, convergence rates of Markov processes and weak Poincaré inequalities

Some analytic and probabilistic properties of the weak Poincaré inequality are obtained. In particular, for strong Feller Markov processes the existence of this inequality is equivalent to each of the following: (i) the Liouville property (or the irreducibility); (ii) the existence of successful couplings (or shift-couplings); (iii) the convergence of the Markov process in total variation norm; (iv) the triviality of the tail (or the invariant) σ-field; (v) the convergence of the density. Estimates of the convergence rate in total variation norm of Markov processes are obtained using the weak Poincaré inequality     

Exponential bounds for convergence of entropy rate approximations in hidden Markov models satisfying a path-mergeability condition

A hidden Markov model (HMM) is said to have path-mergeable states   if for any two states i,j$i,j$ there exist a word w$w$ and state k$k$ such that it is possible to transition from both i$i$ and j$j$ to k$k$ while emitting w$w$. We show that for a finite HMM with path-mergeable states the block estimates of the entropy rate converge exponentially fast. We also show that the path-mergeability property is asymptotically typical in the space of HMM topologies and easily testable.     

Ordered thinnings of point processes and random measures

This is a study of thinnings of point processes and random measures on the real line that satisfy a weak law of large numbers. The thinning procedures have dependencies based on the order of the points or masses being thinned such that the thinned process is a composition of two random measures. It is shown that the thinned process (normalized by a certain function) converges in distribution if and only if the thinning process does. This result is used to characterize the convergence of thinned processes to infinitely divisible processes, such as a compound Poisson process, when the thinning is independent and nonhomogeneous, stationary, Markovian, or regenerative. Thinning by a sequence of independent identically distributed operations is also discussed. The results here contain Renyi's classical thinning theorem and many of its extensions.     

Stochastic approximations of constrained discounted Markov decision processes

We consider a discrete-time constrained Markov decision process under the discounted cost optimality criterion. The state and action spaces are assumed to be Borel spaces, while the cost and constraint functions might be unbounded. We are interested in approximating numerically the optimal discounted constrained cost. To this end, we suppose that the transition kernel of the Markov decision process is absolutely continuous with respect to some probability measure μ  . Then, by solving the linear programming formulation of a constrained control problem related to the empirical probability measure _μn${\mu }_n$ of μ, we obtain the corresponding approximation of the optimal constrained cost. We derive a concentration inequality which gives bounds on the probability that the estimation error is larger than some given constant. This bound is shown to decrease exponentially in n. Our theoretical results are illustrated with a numerical application based on a stochastic version of the Beverton–Holt population model.     

Non-ergodicity criteria for denumerable continuous time Markov processes

We provide non-ergodicity criteria for denumerable continuous time Markov processes in terms of test functions. Two examples are given where the non-ergodicity criteria are applied.     

The convergence of matrix transforms for certain Markov chains

In this paper it is shown that, for certain classes of matrices, the matrix transform of a periodic strongly ergodic stochastic matrix P converges.     

Wasserstein convergence of invariant measures for fractional stochastic reaction–diffusion equations on unbounded domains

This paper is concerned with the convergence of invariant measures in the Wasserstein sense for fractional stochastic reaction–diffusion equations defined on unbounded domains as the noise intensity approaches zero. Based on uniform estimates of solutions, we prove the family of invariant measures of the stochastic equations converges to the invariant measure of the corresponding deterministic equations in terms of the Wasserstein metric. We also provide the rate of such convergence.     

On an extremal property of Markov chains and sufficiency of Markov strategies in Markov decision processes with the Dubins-Savage criterion

An inequality regarding the minimum ofP(lim inf(X _n D _n )) is proved for a class of random sequences. This result is related to the problem of sufficiency of Markov strategies for Markov decision processes with the Dubins-Savage criterion, the asymptotical behaviour of nonhomogeneous Markov chains, and some other problems.