Some stochastic properties of “semi-magic” and “magic” Markov chains

Gustafson and Styan (Gustafson and Styan, Superstochastic matrices and Magic Markov chains, Linear Algebra Appl. 430 (2009) 2705-2715) examined the mathematical properties of superstochastic matrices, the transition matrices of “magic” Markov chains formed from scaled “magic squares”. This paper explores the main stochastic properties of such chains as well as “semi-magic” chains (with doubly-stochastic transition matrices). Stationary distribution, generalized inverses of Markovian kernels, mean first passage times, variances of the first passage times and expected times to mixing are considered. Some general results are developed, some observations from the chains generated by MATLAB are discussed, some conjectures are presented and some special cases, involving three and four states, are explored in detail.     

Markov snakes and superprocesses

Summary We suggest the name Markov snakes for a class of path-valued Markov processes introduced recently by J.-F. Le Gall in connection with the theory of branching measure-valued processes. Le Gall applied this class to investigate path properties of superdiffusions and to approach probabilistically partial differential equations involving a nonlinear operator v–v ². We establish an isomorphism theorem which allows to translate results on continuous superprocesses into the language of Markov snakes and vice versa. By using this theorem, we get limit theorems for discrete Markov snakes.Partially supported by National Science Foundation Grant DMS-9301315 and by The US Army Research Office through the Mathematical Sciences Institute at Cornell University     

An invariance principle for non-symmetric Markov processes and reflecting diffusions in random domains

Summary We study an invariance principle for additive functionals of nonsymmetric Markov processes with singular mean forward velocities. We generalize results of Kipnis and Varadhan [KV] and De Masi et al. [De] in two directions: Markov processes are non-symmetric, and mean forward velocities are distributions. We study continuous time Markov processes. We use our result to homogenize non-symmetric reflecting diffusions in random domains.     

The law of iterated logarithm for additive functionals and martingale additive functionals of Harris recurrent Markov processes

We use Nummelin splitting in continuous time in order to prove laws of iterated logarithm for additive functionals of a Harris recurrent Markov process, with deterministic or random renormalization.     

Analytic characterization of conditional gaugeability for non-local Feynman-Kac transforms

An analytic characterization of gaugeability and conditional gaugeability is given for non-local (or discontinuous) Feynman-Kac transforms of general symmetric Markov processes. This analytic characterization is very useful in determining whether a process perturbed by a potential is gaugeable or conditionally gaugeable in concrete cases.     

Further results for semiregenerative phenomena

A theory of semiregenerative phenomena was developed by the author. The set of points at which such a phenomenon occurs is called a semi regenerative set. There is a correspondence between a semiregenerative set and the range of a Markov subordinator with a unit drift (or a Markov renewal process in the discrete time case). Prabhu, Tang, and Zhu showed that the properties of semiregenerative sets associated with Markov random walks completely characterize the fluctuation behaviour of these processes in the nondegenerate case and also established a Wiener-Hopf factorization based on these sets. These results are surveyed in this paper.     

On permanental processes

Permanental processes can be viewed as a generalization of squared centered Gaussian processes. We analyze the connections of these processes with the local time process of general Markov processes. The obtained results are related to the notion of infinite divisibility.     

On geometric and algebraic transience for discrete-time Markov chains

General characterizations of ergodic Markov chains have been developed in considerable detail. In this paper, we study the transience for discrete-time Markov chains on general state spaces, including the geometric transience and algebraic transience. Criteria are presented through bounding the modified moment of the first return time and establishing the appropriate drift condition. Moreover, we apply the criteria to the random walk on the half line and the skip-free chain on nonnegative integers.     

Forward and reverse representations for Markov chains

In this paper we carry over the concept of reverse probabilistic representations developed in Milstein, Schoenmakers, Spokoiny [G.N. Milstein, J.G.M. Schoenmakers, V. Spokoiny, Transition density estimation for stochastic differential equations via forward–reverse representations, Bernoulli 10 (2) (2004) 281–312] for diffusion processes, to discrete time Markov chains. We outline the construction of reverse chains in several situations and apply this to processes which are connected with jump–diffusion models and finite state Markov chains. By combining forward and reverse representations we then construct transition density estimators for chains which have root-N$N$ accuracy in any dimension and consider some applications.     

Multiscale diffusion approximations for stochastic networks in heavy traffic

Stochastic networks with time varying arrival and service rates and routing structure are studied. Time variations are governed by, in addition to the state of the system, two independent finite state Markov processes X and Y. The transition times of X are significantly smaller than typical inter-arrival and processing times whereas the reverse is true for the Markov process Y. By introducing a suitable scaling parameter one can model such a system using a hierarchy of time scales. Diffusion approximations for such multiscale systems are established under a suitable heavy traffic condition. In particular, it is shown that, under certain conditions, properly normalized buffer content processes converge weakly to a reflected diffusion. The drift and diffusion coefficients of this limit model are functions of the state process, the invariant distribution of X, and a finite state Markov process which is independent of the driving Brownian motion.     

Stochastic processes with orthogonal polynomial eigenfunctions

Markov processes which are reversible with either Gamma, Normal, Poisson or Negative Binomial stationary distributions in the Meixner class and have orthogonal polynomial eigenfunctions are characterized as being processes subordinated to well-known diffusion processes for the Gamma and Normal, and birth and death processes for the Poisson and Negative Binomial. A characterization of Markov processes with Beta stationary distributions and Jacobi polynomial eigenvalues is also discussed.     

On Nummelin splitting for continuous time Harris recurrent Markov processes and application to kernel estimation for multi-dimensional diffusions

We introduce a sequence of stopping times that allow us to study an analogue of a life-cycle decomposition for a continuous time Markov process, which is an extension of the well-known splitting technique of Nummelin to the continuous time case. As a consequence, we are able to give deterministic equivalents of additive functionals of the process and to state a generalisation of Chen’s inequality. We apply our results to the problem of non-parametric kernel estimation of the drift of multi-dimensional recurrent, but not necessarily ergodic, diffusion processes.     

Small mass asymptotic for the motion with vanishing friction

We consider the small mass asymptotic (Smoluchowski–Kramers approximation) for the Langevin equation with a variable friction coefficient. The friction coefficient is assumed to be vanishing within certain region. We introduce a regularization for this problem and study the limiting motion for the 1-dimensional case and a multidimensional model problem. The limiting motion is a Markov process on a projected space. We specify the generator and the boundary condition of this limiting Markov process and prove the convergence.     

A class of path-valued Markov processes and its applications to superprocesses

Summary Let ( _s ) be a continuous Markov process satisfying certain regularity assumptions. We introduce a path-valued strong Markov process associated with ( _s ), which is closely related to the so-called superprocess with spatial motion ( _s ). In particular, a subsetH of the state space of ( _s ) intersects the range of the superprocess if and only if the set of paths that hitH is not polar for the path-valued process. The latter property can be investigated using the tools of the potential theory of symmetric Markov processes: A set is not polar if and only if it supports a measure of finite energy. The same approach can be applied to study sets that are polar for the graph of the superprocess. In the special case when ( _s ) is a diffusion process, we recover certain results recently obtained by Dynkin.     

Geometric ergodicity for classes of homogeneous Markov chains

The paper deals with non asymptotic computable bounds for the geometric convergence rate of homogeneous ergodic Markov processes. Some sufficient conditions are stated for simultaneous geometric ergodicity of Markov chain classes. This property is applied to nonparametric estimation in ergodic diffusion processes.     

Explosion,implosion, and moments of passage times for continuous-time Markov chains: A semimartingale approach

We establish general theorems quantifying the notion of recurrence–through an estimation of the moments of passage times–for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of occurrence of the phenomenon of explosion are also obtained. A new phenomenon of implosion is introduced and sharp conditions for its occurrence are proven. The general results are illustrated by treating models having a difficult behaviour even in discrete time.     

Conditional law of risk processes given that ruin occurs

A risk process that can be Markovised is conditioned on ruin. We prove that the process remains a Markov process. If the risk process is a PDMP, it is shown that the conditioned process remains a PDMP. For many examples the asymptotics of the parameters in both the light-tailed case and the heavy-tailed case are discussed.     

Continuity of non-stationary transition matrices

Conditions are given implying the continuity of the transition functions in a standard Markov transition matrix. An example of a standard transition matrix with discontinuous transition functions is presented and its relation to the Kolmogorov differential equations is discussed.     

Coupling and strong Feller for jump processes on Banach spaces

By using lower bound conditions of the Lévy measure w.r.t. a nice reference measure, the coupling and strong Feller properties are investigated for the Markov semigroup associated with a class of linear SDEs driven by (non-cylindrical) Lévy processes on a Banach space. Unlike in the finite-dimensional case where these properties have also been confirmed for Lévy processes without drift, in the infinite-dimensional setting the appearance of a drift term is essential to ensure the quasi-invariance of the process by shifting the initial data. Gradient estimates and exponential convergence are also investigated. The main results are illustrated by specific models on the Wiener space and separable Hilbert spaces.