Regeneration for chains with infinite memory

Summary A regeneration structure is established for chains with infinite memory. The memory is required to decay only along a single recurrent path. When there are many recurrent paths (e.g. under conservativity) the construction yields a decomposition into regenerative recurrent classes.Research supported by NSF Grant DMS 89-01464     

Crossing and comparison of regenerative processes

A method for a quantitative comparison of wide sense regenerative processes is discussed. The main idea appears to be to make assumptions on the processes being studied that permit one to construct so-called crossing times which are simultaneous regeneration times for another pair of regenerative processes (called crossing), each element of the pair coinciding in distribution with one of the initial processes. Provided that intercrossing times have proper moments (higher, than of the first order), the problem of uniform-in-time comparison is reduced (using renewal-type arguments) to obtaining comparison estimates over finite horizons only. Respective estimates are formulated in terms of probability metrics. Possible applications include continuity of queues, approximation of Markov chains, etc.     

Limit theorems for path-functionals of regenerative processes

Regenerative processes were defined and investigated by Smith [12]. These processes have limiting distributions under very mild regularity conditions. In certain applications, such as shot-noise processes and some queueing problems, it is of interest to consider path-functionals of regenerative processes. We seek to extend the nice asymptotic properties of regenerative processes to path-functionals of regenerative processes. We show that these more general processes converge to a “steady-state” process in a certain weak sense. This is applied to show convergence of shot-noise processes. We also present a Blackwell theorem for path-functionals of regenerative processes.     

Fluctuations of empirical means at low temperature for finite Markov chains with rare transitions in the general case

Summary. The integrated autocovariance and autocorrelation time are essential tools to understand the dynamical behavior of a Markov chain. We study here these two objects for Markov chains with rare transitions with no reversibility assumption. We give upper bounds for the autocovariance and the integrated autocorrelation time, as well as exponential equivalents at low temperature. We also link their slowest modes with the underline energy landscape under mild assumptions. Our proofs will be based on large deviation estimates coming from the theory of Wentzell and Freidlin and others [4, 3, 12], and on coupling arguments (see [6] for a review on the coupling method). Received 5 August 1996 / In revised form: 6 August 1997     

Forward and reverse representations for Markov chains

In this paper we carry over the concept of reverse probabilistic representations developed in Milstein, Schoenmakers, Spokoiny [G.N. Milstein, J.G.M. Schoenmakers, V. Spokoiny, Transition density estimation for stochastic differential equations via forward–reverse representations, Bernoulli 10 (2) (2004) 281–312] for diffusion processes, to discrete time Markov chains. We outline the construction of reverse chains in several situations and apply this to processes which are connected with jump–diffusion models and finite state Markov chains. By combining forward and reverse representations we then construct transition density estimators for chains which have root-N$N$ accuracy in any dimension and consider some applications.     

Further results for semiregenerative phenomena

A theory of semiregenerative phenomena was developed by the author. The set of points at which such a phenomenon occurs is called a semi regenerative set. There is a correspondence between a semiregenerative set and the range of a Markov subordinator with a unit drift (or a Markov renewal process in the discrete time case). Prabhu, Tang, and Zhu showed that the properties of semiregenerative sets associated with Markov random walks completely characterize the fluctuation behaviour of these processes in the nondegenerate case and also established a Wiener-Hopf factorization based on these sets. These results are surveyed in this paper.     

Berry–Esseen estimates for regenerative processes under weak moment assumptions

We prove Berry–Esseen type rates of convergence for central limit theorems (CLTs) of regenerative processes which generalize previous results of Bolthausen under weaker moment assumptions. We then show how this general result can be applied to obtain rates of convergence for (1) CLTs for additive functionals of positive recurrent Markov chains under certain conditions on the strong mixing coefficients, and (2) annealed CLTs for certain ballistic random walks in random environments.     

Geometric ergodicity for classes of homogeneous Markov chains

The paper deals with non asymptotic computable bounds for the geometric convergence rate of homogeneous ergodic Markov processes. Some sufficient conditions are stated for simultaneous geometric ergodicity of Markov chain classes. This property is applied to nonparametric estimation in ergodic diffusion processes.     

Itô’s stochastic calculus and Heisenberg commutation relations

Stochastic calculus and stochastic differential equations for Brownian motion were introduced by K. Itô in order to give a pathwise construction of diffusion processes. This calculus has deep connections with objects such as the Fock space and the Heisenberg canonical commutation relations, which have a central role in quantum physics. We review these connections, and give a brief introduction to the noncommutative extension of Itô’s stochastic integration due to Hudson and Parthasarathy. Then we apply this scheme to show how finite Markov chains can be constructed by solving stochastic differential equations, similar to diffusion equations, on the Fock space.     

Law of the iterated logarithm for the periodogram

We consider the almost sure asymptotic behavior of the periodogram of stationary and ergodic sequences. Under mild conditions we establish that the limsup of the periodogram properly normalized identifies almost surely the spectral density function associated with the stationary process. Results for a specified frequency are also given. Our results also lead to the law of the iterated logarithm for the real and imaginary parts of the discrete Fourier transform. The proofs rely on martingale approximations combined with results from harmonic analysis and techniques from ergodic theory. Several applications to linear processes and their functionals, iterated random functions, mixing structures and Markov chains are also presented.     

Explosion,implosion, and moments of passage times for continuous-time Markov chains: A semimartingale approach

We establish general theorems quantifying the notion of recurrence–through an estimation of the moments of passage times–for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of occurrence of the phenomenon of explosion are also obtained. A new phenomenon of implosion is introduced and sharp conditions for its occurrence are proven. The general results are illustrated by treating models having a difficult behaviour even in discrete time.     

On the convergence to the multiple Wiener-Itô integral

We study the convergence to the multiple Wiener-Itô integral from processes with absolutely continuous paths. More precisely, consider a family of processes, with paths in the Cameron-Martin space, that converges weakly to a standard Brownian motion in C₀([0,T]). Using these processes, we construct a family that converges weakly, in the sense of the finite dimensional distributions, to the multiple Wiener-Itô integral process of a function f∈L²(n[0,T]). We prove also the weak convergence in the space C₀([0,T]) to the second-order integral for two important families of processes that converge to a standard Brownian motion.     

Uniform ergodicities and perturbation bounds of Markov chains on base norm spaces

It is known that Dobrushin's ergodicity coefficient is one of the effective tools in the investigations of limiting behavior of Markov processes. Several interesting properties of the ergodicity coefficient of a positive mapping defined on base norm spaces have been studied. In this paper, we consider uniformly mean ergodic and asymptotically stable Markov operators on such spaces. In terms of the ergodicity coefficient, we establish uniform mean ergodicity criterion. Moreover, we develop the perturbation theory for uniformly asymptotically stable Markov chains on base norm spaces. In particularly, main results open new perspectives in the perturbation theory for quantum Markov processes defined on von Neumann algebras.     

Multiscale diffusion approximations for stochastic networks in heavy traffic

Stochastic networks with time varying arrival and service rates and routing structure are studied. Time variations are governed by, in addition to the state of the system, two independent finite state Markov processes X and Y. The transition times of X are significantly smaller than typical inter-arrival and processing times whereas the reverse is true for the Markov process Y. By introducing a suitable scaling parameter one can model such a system using a hierarchy of time scales. Diffusion approximations for such multiscale systems are established under a suitable heavy traffic condition. In particular, it is shown that, under certain conditions, properly normalized buffer content processes converge weakly to a reflected diffusion. The drift and diffusion coefficients of this limit model are functions of the state process, the invariant distribution of X, and a finite state Markov process which is independent of the driving Brownian motion.     

Large deviations for renewal processes

We investigate large deviations for the empirical measure of the forward and backward recurrence time processes associated with a classical renewal process with arbitrary waiting-time distribution. The Donsker-Varadhan theory cannot be applied in this case, and indeed it turns out that the large deviations rate functional differs from the one suggested by such a theory. In particular, a non-strictly convex and non-analytic rate functional is obtained.     

Stationarity and geometric ergodicity of BEKK multivariate GARCH models

Conditions for the existence of strictly stationary multivariate GARCH processes in the so-called BEKK parametrisation, which is the most general form of multivariate GARCH processes typically used in applications, and for their geometric ergodicity are obtained. The conditions are that the driving noise is absolutely continuous with respect to the Lebesgue measure and zero is in the interior of its support and that a certain matrix built from the GARCH coefficients has spectral radius smaller than one.To establish the results, semi-polynomial Markov chains are defined and analysed using algebraic geometry.     

Empirical processes of multidimensional systems with multiple mixing properties

We establish a multivariate empirical process central limit theorem for stationary R^d-valued stochastic processes (X_i)_i≥1 under very weak conditions concerning the dependence structure of the process. As an application, we can prove the empirical process CLT for ergodic torus automorphisms. Our results also apply to Markov chains and dynamical systems having a spectral gap on some Banach space of functions. Our proof uses a multivariate extension of the techniques introduced by Dehling et al. (2009) [9] in the univariate case. As an important technical ingredient, we prove a 2pth moment bound for partial sums in multiple mixing systems.     

Extending the Wong-Zakai theorem to reversible Markov processes

We show how to construct a canonical choice of stochastic area for paths of reversible Markov processes satisfying a weak H?lder condition, and hence demonstrate that the sample paths of such processes are rough paths in the sense of Lyons. We further prove that certain polygonal approximations to these paths and their areas converge in p-variation norm. As a corollary of this result and standard properties of rough paths, we are able to provide a significant generalization of the classical result of Wong-Zakai on the approximation of solutions to stochastic differential equations. Our results allow us to construct solutions to differential equations driven by reversible Markov processes of finite p-variation with p<4. Received May 18, 2001 / final version received April 3, 2001?Published online April 8, 2002     

Coupling and convergence of random elements,processes, and regenerative processes

The paper starts by proving that a sequence of random elements can be coupled in such a way that the random elements eventually coincide if and only if liminf of their densities is a density. It continues with a survey of some general coupling theory for stochastic processes and applications to wide sense regenerative processes and Palm theory. Finally, a successful coupling and -coupling of wide sense regenerative processes is constructed without assuming that the inter-regeneration times have finite mean.     

Fluctuations of interacting Markov chain Monte Carlo methods

We present a multivariate central limit theorem for a general class of interacting Markov chain Monte Carlo algorithms used to solve nonlinear measure-valued equations. These algorithms generate stochastic processes which belong to the class of nonlinear Markov chains interacting with their empirical occupation measures. We develop an original theoretical analysis based on resolvent operators and semigroup techniques to analyze the fluctuations of their occupation measures around their limiting values.