Spectral computations for birth and death chains

We consider the spectrum of birth and death chains on an n$n$-path. An iterative scheme is proposed to compute any eigenvalue with exponential convergence rate independent of n$n$. This allows one to determine the whole spectrum in order n²$n^2$ elementary operations. Using the same idea, we also provide a lower bound on the spectral gap, which is of the correct order on some classes of examples.     

Large deviations and related problems for absorbing Markov chains

In this paper, large deviations and their connections with several other fundamental topics are investigated for absorbing Markov chains. A variational representation for the Dirichlet principal eigenvalues is given by the large deviation approach. Kingman’s decay parameters and mean ratio quasi-stationary distributions of the chains are also characterized by the large deviation rate function. As an application of these results, we interpret the “stationarity” of mean ratio quasi-stationary distributions via a concrete example. An application to quasi-ergodicity is also discussed.     

Cutoffs for product chains

We consider products of ergodic Markov chains and discuss their cutoffs in total variation. Our framework is general in that rates to pick up coordinates are not necessary equal, and different coordinates may correspond to distinct chains. We give necessary and sufficient conditions for cutoffs of product chains in terms of those of coordinate chains under certain conditions. A comparison of mixing times between the product chain and its coordinate chains is made in detail as well. Examples are given to show that neither cutoffs for product chains nor for coordinate chains imply others in general.     

Entrywise perturbation theory and error analysis for Markov chains

Summary Grassmann, Taksar, and Heyman introduced a variant of Gaussian climination for computing the steady-state vector of a Markov chain. In this paper we prove that their algorithm is stable, and that the problem itself is well-conditioned, in the sense of entrywise relative error. Thus the algorithm computes each entry of the steady-state vector with low relative error. Even the small steady-state probabilities are computed accurately. The key to our analysis is to focus on entrywise relative error in both the data and the computed solution, rather than making the standard assessments of error based on norms. Our conclusions do not depend on any Condition numbers for the problem.This work was supported by NSF under grants DMS-9106207 and DDM-9203134     

Geometric ergodicity for classes of homogeneous Markov chains

The paper deals with non asymptotic computable bounds for the geometric convergence rate of homogeneous ergodic Markov processes. Some sufficient conditions are stated for simultaneous geometric ergodicity of Markov chain classes. This property is applied to nonparametric estimation in ergodic diffusion processes.     

Syzygies for Metropolis base chains

It is shown that Markov chains for sampling from combinatorial sets in the form of experimental designs can be made more efficient by using syzygies on gradient vectors. Examples are presented.     

On geometric and algebraic transience for discrete-time Markov chains

General characterizations of ergodic Markov chains have been developed in considerable detail. In this paper, we study the transience for discrete-time Markov chains on general state spaces, including the geometric transience and algebraic transience. Criteria are presented through bounding the modified moment of the first return time and establishing the appropriate drift condition. Moreover, we apply the criteria to the random walk on the half line and the skip-free chain on nonnegative integers.     

Some stochastic properties of “semi-magic” and “magic” Markov chains

Gustafson and Styan (Gustafson and Styan, Superstochastic matrices and Magic Markov chains, Linear Algebra Appl. 430 (2009) 2705-2715) examined the mathematical properties of superstochastic matrices, the transition matrices of “magic” Markov chains formed from scaled “magic squares”. This paper explores the main stochastic properties of such chains as well as “semi-magic” chains (with doubly-stochastic transition matrices). Stationary distribution, generalized inverses of Markovian kernels, mean first passage times, variances of the first passage times and expected times to mixing are considered. Some general results are developed, some observations from the chains generated by MATLAB are discussed, some conjectures are presented and some special cases, involving three and four states, are explored in detail.     

A martingale decomposition for quadratic forms of Markov chains (with applications)

We develop a martingale-based decomposition for a general class of quadratic forms of Markov chains, which resembles the well-known Hoeffding decomposition of U$U$-statistics of i.i.d. data up to a reminder term. To illustrate the applicability of our results, we discuss how this decomposition may be used to studying the large-sample properties of certain statistics in two problems: (i) we examine the asymptotic behavior of lag-window estimators in time series, and (ii) we derive an asymptotic linear representation and limiting distribution of U$U$-statistics with varying kernels in time series. We also discuss simplified examples of interest in statistics and econometrics.     

Isoperimetric Invariants For Product Markov Chains and Graph Products

Bounds on some isoperimetric constants of the Cartesian product of Markov chains are obtained in terms of related isoperimetric quantities of the individual chains.* Research supported in part by NSF Grants. Research supported by NSF Grant No. CCR-9503952 and DMS-9800351.     

Explosion,implosion, and moments of passage times for continuous-time Markov chains: A semimartingale approach

We establish general theorems quantifying the notion of recurrence–through an estimation of the moments of passage times–for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of occurrence of the phenomenon of explosion are also obtained. A new phenomenon of implosion is introduced and sharp conditions for its occurrence are proven. The general results are illustrated by treating models having a difficult behaviour even in discrete time.     

Potential analysis for positive recurrent Markov chains with asymptotically zero drift: Power-type asymptotics

We consider a positive recurrent Markov chain on R⁺${\mathbb{R}}^{+}$ with asymptotically zero drift which behaves like −_c1/x$-c_1/x$ at infinity; this model was first considered by Lamperti. We are interested in tail asymptotics for the stationary measure. Our analysis is based on construction of a harmonic function which turns out to be regularly varying at infinity. This harmonic function allows us to perform non-exponential change of measure. Under this new measure Markov chain is transient with drift like _c2/x$c_2/x$ at infinity and we compute the asymptotics for its Green function. Applying further the inverse transform of measure we deduce a power-like asymptotic behaviour of the stationary tail distribution. Such a heavy-tailed stationary measure happens even if the jumps of the chain are bounded. This model provides an example where possibly bounded input distributions produce non-exponential output.     

The cutoff phenomenon for random birth and death chains

For any distribution π on , we study elements drawn at random from the set of tridiagonal stochastic matrices K satisfying for all . These matrices correspond to birth and death chains with stationary distribution π. We analyze an algorithm for sampling from and use results from this analysis to draw conclusions about the Markov chains corresponding to typical elements of . Our main interest is in determining when certain sequences of random birth and death chains exhibit the cutoff phenomenon. © 2016 Wiley Periodicals, Inc. Random Struct. Alg., 50, 287–321, 2017     

Strong approximation theorems for density dependent Markov chains

A variety of continuous parameter Markov chains arising in applied probability (e.g. epidemic and chemical reaction models) can be obtained as solutions of equations of the form

$\text{X}{}_{\mathrm{N}}\text{(t)=x}{}_0\text{+∑}\text{1}\text{N}\text{lY}{}_1\text{N ∫}{}^{\mathrm{t}}{}_0\text{f}{}_1\text{(X}{}_{\mathrm{N}}\text{(s))ds}$

where $\text{l∈}\text{Z}{}^{\mathrm{t}}$, the Y₁ are independent Poisson processes, and N is a parameter with a natural interpretation (e.g. total population size or volume of a reacting solution).The corresponding deterministic model, satisfies

$\text{X(t)=x}{}_0\text{+ ∫}{}^{\mathrm{t}}{}_0\text{∑ lf}{}_1\text{(X(s))ds}$

Under very general conditions lim_N→∞X_N(t)=X(t) a.s. The process X_N(t) is compared to the diffusion processes given by

$\text{Z}{}_{\mathrm{N}}\text{(t)=x}{}_0\text{+∑}\text{1}\text{N}\text{lB}{}_1\text{N∫}{}^{\mathrm{t}}{}_0\text{ft(Z}{}_{\mathrm{N}}\text{(s))ds}$

and

$\text{V(t)=∑ l∫}{}^{\mathrm{t}}{}_0\text{f}{}_1\text{(X(s))}\text{d}\text{W}\text{?}{}_1\text{+∫}{}^{\mathrm{t}}{}_0\text{?F(X(s))·V(s)}\text{ds.}$

Under conditions satisfied by most of the applied probability models, it is shown that X_N,Z_N and V can be constructed on the same sample space in such a way that

$\text{X}{}_{\mathrm{N}}\text{(t)=Z}{}_{\mathrm{N}}\text{(t)+O}\text{logN}\text{N}$

and

$\text{N}\text{(X}{}_{\mathrm{N}}\text{(t)?X(t))=V(t)+O}\text{log N}\text{N}$

     

New perturbation bounds for denumerable Markov chains

This paper is devoted to perturbation analysis of denumerable Markov chains. Bounds are provided for the deviation between the stationary distribution of the perturbed and nominal chain, where the bounds are given by the weighted supremum norm. In addition, bounds for the perturbed stationary probabilities are established. Furthermore, bounds on the norm of the asymptotic decomposition of the perturbed stationary distribution are provided, where the bounds are expressed in terms of the norm of the ergodicity coefficient, or the norm of a special residual matrix. Refinements of our bounds for Doeblin Markov chains are considered as well. Our results are illustrated with a number of examples.     

Forward and reverse representations for Markov chains

In this paper we carry over the concept of reverse probabilistic representations developed in Milstein, Schoenmakers, Spokoiny [G.N. Milstein, J.G.M. Schoenmakers, V. Spokoiny, Transition density estimation for stochastic differential equations via forward–reverse representations, Bernoulli 10 (2) (2004) 281–312] for diffusion processes, to discrete time Markov chains. We outline the construction of reverse chains in several situations and apply this to processes which are connected with jump–diffusion models and finite state Markov chains. By combining forward and reverse representations we then construct transition density estimators for chains which have root-N$N$ accuracy in any dimension and consider some applications.     

Imprecise Markov chains with absorption

We consider convergence of Markov chains with uncertain parameters, known as imprecise Markov chains, which contain an absorbing state. We prove that under conditioning on non-absorption the imprecise conditional probabilities converge independently of the initial imprecise probability distribution if some regularity conditions are assumed. This is a generalisation of a known result from the classical theory of Markov chains by Darroch and Seneta [6].     

On asymptotics for Vaserstein coupling of Markov chains

We prove that strong ergodicity of a Markov process is linked with a spectral radius of a certain “associated” semigroup operator, although, not a “natural” one. We also give sufficient conditions for weak ergodicity and provide explicit estimates of the convergence rate. To establish these results we construct a modification of the Vaserstein coupling. Some applications including mixing properties are also discussed.