Ergodicity for countable inhomogeneous Markov chains

A notion of ergodicity is defined by analogy to homogeneous chains, and a necessary and sufficient condition for it to hold for an inhomogeneous Markov chain is given in terms of matrix products. A comparison to the situation for homogeneous chains is made. A final section discusses the better-known notion of strong ergodicity in relation to the geometric convergence rate.     

Pathwise convergence rates for numerical solutions of Markovian switching stochastic differential equations

This work develops numerical approximation algorithms for solutions of stochastic differential equations with Markovian switching. The existing numerical algorithms all use a discrete-time Markov chain for the approximation of the continuous-time Markov chain. In contrast, we generate the continuous-time Markov chain directly, and then use its skeleton process in the approximation algorithm. Focusing on weak approximation, we take a re-embedding approach, and define the approximation and the solution to the switching stochastic differential equation on the same space. In our approximation, we use a sequence of independent and identically distributed (i.i.d.) random variables in lieu of the common practice of using Brownian increments. By virtue of the strong invariance principle, we ascertain rates of convergence in the pathwise sense for the weak approximation scheme.     

Inverse problems for ergodicity of Markov chains

For continuous-time Markov chains, we provide criteria for non-ergodicity, non-algebraic ergodicity, non-exponential ergodicity, and non-strong ergodicity. For discrete-time Markov chains, criteria for non-ergodicity, non-algebraic ergodicity, and non-strong ergodicity are given. Our criteria are in terms of the existence of solutions to inequalities involving the Q-matrix (or transition matrix P in time-discrete case) of the chain. Meanwhile, these practical criteria are applied to some examples, including a special class of single birth processes and several multi-dimensional models.     

On strong ergodicity for nonhomogeneous continuous-time Markov chains

Let X(t) be a nonhomogeneous continuous-time Markov chain. Suppose that the intensity matrices of X(t) and some weakly or strongly ergodic Markov chain (t) are close. Some sufficient conditions for weak and strong ergodicity of X(t) are given and estimates of the rate of convergence are proved. Queue-length for a birth and death process in the case of asymptotically proportional intensities is considered as an example.     

Continuous-time limit of repeated interactions for a system in a confining potential

We study the continuous-time limit of a class of Markov chains coming from the evolution of classical open systems undergoing repeated interactions. This repeated interaction model has been initially developed for dissipative quantum systems in Attal and Pautrat (2006) and was recently set up for the first time in Deschamps (2012) for classical dynamics. It was particularly shown in the latter that this scheme furnishes a new kind of Markovian evolutions based on Hamilton’s equations of motion. The system is also proved to evolve in the continuous-time limit with a stochastic differential equation. We here extend the convergence of the evolution of the system to more general dynamics, that is, to more general Hamiltonians and probability measures in the definition of the model. We also present a natural way to directly renormalize the initial Hamiltonian in order to obtain the relevant process in a study of the continuous-time limit. Then, even if Hamilton’s equations have no explicit solution in general, we obtain some bounds on the dynamics allowing us to prove the convergence in law of the Markov chain on the system to the solution of a stochastic differential equation, via the infinitesimal generators.     

Fuzzy Markov Chains and Decision-Making

Decision-making in an environment of uncertainty and imprecision for real-world problems is a complex task. In this paper it is introduced general finite state fuzzy Markov chains that have a finite convergence to a stationary (may be periodic) solution. The Cesaro average and the -potential for fuzzy Markov chains are defined, then it is shown that the relationship between them corresponds to the Blackwell formula in the classical theory of Markov decision processes. Furthermore, it is pointed out that recurrency does not necessarily imply ergodicity. However, if a fuzzy Markov chain is ergodic, then the rows of its ergodic projection equal the greatest eigen fuzzy set of the transition matrix. Then, the fuzzy Markov chain is shown to be a robust system with respect to small perturbations of the transition matrix, which is not the case for the classical probabilistic Markov chains. Fuzzy Markov decision processes are finally introduced and discussed.     

Characterisation of ergodic upper transition operators

We study ergodicity for upper transition operators: bounded, sub-additive and non-negatively homogeneous transformations of finite-dimensional linear spaces. Ergodicity provides a necessary and sufficient condition for Perron–Frobenius-like convergence behaviour for upper transition operators. It can also be characterised alternatively: (i) using a coefficient of ergodicity, and (ii) using accessibility relations. The latter characterisation states that ergodicity is equivalent with there being a single maximal communication (or top) class that is moreover regular and absorbing. We present an algorithm for checking these conditions that is linear in the dimension of the state space for the number of evaluations of the upper transition operator.     

Geometric ergodicity of the Gibbs sampler for the Poisson change-point model

Poisson change-point models have been widely used for modelling inhomogeneous time-series of count data. There are a number of methods available for estimating the parameters in these models using iterative techniques such as MCMC. Many of these techniques share the common problem that there does not seem to be a definitive way of knowing the number of iterations required to obtain sufficient convergence. In this paper, we show that the Gibbs sampler of the Poisson change-point model is geometrically ergodic. Establishing geometric ergodicity is crucial from a practical point of view as it implies the existence of a Markov chain central limit theorem, which can be used to obtain standard error estimates. We prove that the transition kernel is a trace-class operator, which implies geometric ergodicity of the sampler. We then provide a useful application of the sampler to a model for the quarterly driver fatality counts for the state of Victoria, Australia.     

非齐次马尔科夫链遍历性的一些结果

本文利用两个非齐次马尔科夫链的转移矩阵列的比较，讨论了两个链启遍历性的关系，得到一个非齐次马尔科夫链是强遍历的一些充分条件．本文还分析了非齐次马尔科夫链的一致强、弱遍历性的关系，得到一个非齐次马尔科夫链是一致强遍历的一些充分条件．     

Ergodic backward stochastic difference equations

We consider ergodic backward stochastic differential equations in a discrete time setting, where noise is generated by a finite state Markov chain. We show existence and uniqueness of solutions, along with a comparison theorem. To obtain this result, we use a Nummelin splitting argument to obtain ergodicity estimates for a discrete time Markov chain which hold uniformly under suitable perturbations of its transition matrix. We conclude with an application of this theory to a treatment of an ergodic control problem.     

On ergodicity conditions in a polling model with Markov modulated input and state-dependent routing

A polling system with switchover times and state-dependent server routing is studied. Input flows are modulated by a random external environment. Input flows are ordinary Poisson flows in each state of the environment, with intensities determined by the environment state. Service and switchover durations have exponential laws of probability distribution. A continuous-time Markov chain is introduced to describe the dynamics of the server, the sizes of the queues and the states of the environment. By means of the iterative-dominating method a sufficient condition for ergodicity of the system is obtained for the continuous-time Markov chain. This condition also ensures the existence of a stationary probability distribution of the embedded Markov chain at instants of jumps. The customers sojourn cost during the period of unloading the stable queueing system is chosen as a performance metric. Numerical study in case of two input flows and a class of priority and threshold routing algorithms is conducted. It is demonstrated that in case of light inputs a priority routing rule doesn’t seem to be quasi-optimal.     

On duality for mathematical programs with vanishing constraints

We make a review of several variants of ergodicity for continuous-time Markov chains on a countable state space. These include strong ergodicity, ergodicity in weighted-norm spaces, exponential and subexponential ergodicity. We also study uniform exponential ergodicity for continuous-time controlled Markov chains, as a tool to deal with average reward and related optimality criteria. A discussion on the corresponding ergodicity properties is made, and an application to a controlled population system is shown.     

Finite-dimensional approximations for the equation of nonlinear filtering derived in mild form

The Zakai equation for the unnormalized conditional density is derived as a mild stochastic bilinear differential equation on a suitableL ² space. It is assumed that the Markov semigroup corresponding to the state process isC ₀ on such space. This allows the establishment of the existence and uniqueness of the solution by means of general theorems on stochastic differential equations in Hilbert space. Moreover, an easy treatment of convergence conditions can be given for a general class of finite-dimensional approximations, including Galerkin schemes. This is done by using a general continuity result for the solution of a mild stochastic bilinear differential equation on a Hilbert space with respect to the semigroup, the forcing operator, and the initial state, within a suitable topology.     

Stochastic Detectability and Mean Bounded Error Covariance of the Recursive Kalman Filter with Markov Jump Parameters

In this article, we study the error covariance of the recursive Kalman filter when the parameters of the filter are driven by a Markov chain taking values in a countably infinite set. We do not assume ergodicity nor require the existence of limiting probabilities for the Markov chain. The error covariance matrix of the filter depends on the Markov state realizations, and hence forms a stochastic process. We show in a rather direct and comprehensive manner that this error covariance process is mean bounded under the standard stochastic detectability concept. Illustrative examples are included.     

On the optimality equation for average cost Markov control processes with Feller transition probabilities

We consider Markov control processes with Borel state space and Feller transition probabilities, satisfying some generalized geometric ergodicity conditions. We provide a new theorem on the existence of a solution to the average cost optimality equation.     

Nonlinear discretization errors in partial difference equations

A necessary and sufficient condition for convergence of a finite difference solution of a nonlinear partial differential equation to its exact solution, assumed to exist, has been established in this work.     

Geometric Ergodicity for Piecewise Contracting Processes with Applications for Tropical Stochastic Lattice Models

Stochastic lattice models are increasingly prominent as a way to capture highly intermittent unresolved features of moist tropical convection in climate science and as continuum mesoscopic models in material science. Stochastic lattice models consist of suitably discretized continuum partial differential equations interacting with Markov jump processes at each lattice site with transition rates depending on the local value of the continuum equation; they are a special case of piecewise deterministic Markov processes but often have an infinite state space and unbounded transition rates. Here a general theorem on geometric ergodicity for piecewise deterministic contracting processes is developed with full generality to apply to stochastic lattice models. A highly nontrivial application to the stochastic skeleton model for the Madden‐Julian oscillation (Thual et al., 2013) is developed here where there is an infinite state space with unbounded and also degenerate transition rates. Geometric ergodicity for the stochastic skeleton model guarantees exponential convergence to a unique invariant measure that defines the statistical tropical climate of the model. Another application of the general framework is developed here for stochastic lattice models designed to capture intermittent fluctuation in the simplest tropical climate models. Other straightforward applications to models motivated by material science are mentioned briefly here. © 2016 Wiley Periodicals, Inc.     

Bond pricing under a Markovian regime-switching jump-augmented Vasicek model via stochastic flows

In this article, we shall explore the state of art of stochastic flows to derive an exponential affine form of the bond price when the short rate process is governed by a Markovian regime-switching jump-diffusion version of the Vasicek model. We provide the flexibility that the market parameters, including the mean-reversion level, the volatility rate and the intensity of the jump component switch over time according to a continuous-time, finite-state Markov chain. The states of the chain may be interpreted as different states of an economy or different stages of a business cycle. We shall provide a representation for the exponential affine form of the bond price in terms of fundamental matrix solutions of linear matrix differential equations.     

On asymptotics for Vaserstein coupling of Markov chains

We prove that strong ergodicity of a Markov process is linked with a spectral radius of a certain “associated” semigroup operator, although, not a “natural” one. We also give sufficient conditions for weak ergodicity and provide explicit estimates of the convergence rate. To establish these results we construct a modification of the Vaserstein coupling. Some applications including mixing properties are also discussed.     

信息熵的收敛性与收敛速度

本讨论了连续时间，状态空间有限的马氏过程的信息熵的收敛性质，给出了可测转移矩阵的极限形式，得出具有可测转移矩阵的马氏过程的信息熵在时间趋于无穷大时存在且有限，对于具有强遍历转移函数的马氏过程的信息熵按多项式一致收敛。