Chutes and Ladders in Markov Chains

We investigate how the stationary distribution of a Markov chain changes when transitions from a single state are modified. In particular, adding a single directed edge to nearest neighbor random walk on a finite discrete torus in dimensions one, two, or three changes the stationary distribution linearly, logarithmically, or only locally. Related results are derived for birth and death chains approximating Bessel diffusions and for random walk on the Sierpinski gasket.     

A random environment for linearly edge-reinforced random walks on infinite graphs

We consider linearly edge-reinforced random walk on an arbitrary locally finite connected graph. It is shown that the process has the same distribution as a mixture of reversible Markov chains, determined by time-independent strictly positive weights on the edges. Furthermore, we prove bounds for the random weights, uniform, among others, in the size of the graph.      

On Transience Conditions for Markov Chains and Random Walks

We prove a new transience criterion for Markov chains on an arbitrary state space and give a corollary for real-valued chains. We show by example that in the case of a homogeneous random walk with infinite mean the proposed sufficient conditions are close to those necessary. We give a new proof of the well-known criterion for finiteness of the supremum of a random walk.     

Gumbel fluctuations for cover times in the discrete torus

This work proves that the fluctuations of the cover time of simple random walk in the discrete torus of dimension at least three with large side-length are governed by the Gumbel extreme value distribution. This result was conjectured for example in Aldous and Fill (Reversible Markov chains and random walks on graphs, in preparation). We also derive some corollaries which qualitatively describe “how” covering happens. In addition, we develop a new and stronger coupling of the model of random interlacements, introduced by Sznitman (Ann Math (2) 171(3):2039–2087, 2010), and random walk in the torus. This coupling is used to prove the cover time result and is also of independent interest.     

ON MARKOV CHAINS IN SPACE-TIME RANDOM ENVIRONMENTS

In Section 1, the authors establish the models of two kinds of Markov chains in space-time random environments (MCSTRE and MCSTRE(+)) with abstract state space. In Section 2, the authors construct a MCSTRE and a MCSTRE(+) by an initial distribution Φ and a random Markov kernel (RMK) p(γ). In Section 3, the authors es-tablish several equivalence theorems on MCSTRE and MCSTRE(+). Finally, the authors give two very important examples of MCMSTRE, the random walk in spce-time random environment and the Markov br...     

Two-boundary problems for a random walk

We solve main two-boundary problems for a random walk. The generating function of the joint distribution of the first exit time of a random walk from an interval and the value of the overshoot of the random walk over the boundary at exit time is determined. We also determine the generating function of the joint distribution of the first entrance time of a random walk to an interval and the value of the random walk at this time. The distributions of the supremum, infimum, and value of a random walk and the number of upward and downward crossings of an interval by a random walk are determined on a geometrically distributed time interval. We give examples of application of obtained results to a random walk with one-sided exponentially distributed jumps. __________ Translated from Ukrains’kyi Matematychnyi Zhurnal, Vol. 59, No. 11, pp. 1485–1509, November, 2007.     

Quenched tail estimate for the random walk in random scenery and in random layered conductance

We discuss the quenched tail estimates for the random walk in random scenery. The random walk is the symmetric nearest neighbor walk and the random scenery is assumed to be independent and identically distributed, non-negative, and has a power law tail. We identify the long time asymptotics of the upper deviation probability of the random walk in quenched random scenery, depending on the tail of scenery distribution and the amount of the deviation. The result is in turn applied to the tail estimates for a random walk in random conductance which has a layered structure.     

Light-tailed behavior of stationary distribution for state-dependent random walks on a strip

We consider the state-dependent reflecting random walk on a half- strip. We provide explicit criteria for （positive） recurrence, and an explicit expression for the stationary distribution. As a consequence, the light-tailed behavior of the stationary distribution is proved under appropriate conditions. The key idea of the method employed here is the decomposition of the trajectory of the random walk and the main tool is the intrinsic branching structure buried in the random walk on a strip, which is different from the matrix-analytic method.     

Some results on first passage times in one dimensional random walks

Analytic expressions are presented for the characteristic function of the first passage time distribution for biased random walk on a finite chain (and diffusion with drift on a finite line); of the first passage time distribution for a random walk on a chain, in which the events (jumps) are governed by an arbitrary renewal process; and of the distribution of the time of escape from a bounded set of points in the latter case. A fundamental relation between the first passage time distribution and the conditional probability for random walk (or diffusion) in one dimension is analyzed and generalized.     

Product-form characterization for a two-dimensional reflecting random walk

We consider a two-dimensional skip-free reflecting random walk on the non-negative integers, which is referred to as a 2-d reflecting random walk. We give necessary and sufficient conditions for the stationary distribution to have a product-form. We also derive simpler sufficient conditions for the product-form for a restricted class of 2-d reflecting random walks. We apply these results and obtain a product-form approximation of the stationary distribution through a suitable modification of the parameters of the random walk.     

Edge-reinforced random walk on one-dimensional periodic graphs

In the present paper, linearly edge-reinforced random walk is studied on a large class of one-dimensional periodic graphs satisfying a certain reflection symmetry. It is shown that the edge-reinforced random walk is recurrent. Estimates for the position of the random walker are given. The edge-reinforced random walk has a unique representation as a random walk in a random environment, where the random environment is given by random weights on the edges. It is shown that these weights decay exponentially in space. The distribution of the random weights equals the distribution of the asymptotic proportion of time spent by the edge-reinforced random walker on the edges of the graph. The results generalize work of the authors in Merkl and Rolles (Ann Probab 33(6):2051–2093, 2005; 35(1):115–140, 2007) and Rolles (Probab Theory Related Fields 135(2):216–264, 2006) to a large class of graphs and to periodic initial weights with a reflection symmetry.     

On the Local Time of Random Walks Associated with Gegenbauer Polynomials

The local time of random walks associated with Gegenbauer polynomials \(P_{n}^{(\alpha)}(x)\), x∈[?1,1], is studied in the recurrent case: \(\alpha\in [-\frac{1}{2},0]\). When α is nonzero, the limit distribution is given in terms of a Mittag-Leffler distribution. The proof is based on a local limit theorem for the random walk associated with Gegenbauer polynomials. As a by-product, we derive the limit distribution of the local time of some particular birth-and-death Markov chains on ?.     

Boundaries and Harmonic Functions for Random Walks with Random Transition Probabilities

The usual random walk on a group (homogeneous both in time and in space) is determined by a probability measure on the group. In a random walk with random transition probabilities this single measure is replaced with a stationary sequence of measures, so that the resulting (random) Markov chains are still space homogeneous, but no longer time homogeneous. We study various notions of measure theoretical boundaries associated with this model and establish an analogue of the Poisson formula for (random) bounded harmonic functions. Under natural conditions on transition probabilities we identify these boundaries for several classes of groups with hyperbolic properties and prove the boundary triviality (i.e., the absence of non-constant random bounded harmonic functions) for groups of subexponential growth, in particular, for nilpotent groups.     

The limit distribution of the maximum increment of a random walk with dependent regularly varying jump sizes

We investigate the maximum increment of a random walk with heavy-tailed jump size distribution. Here heavy-tailedness is understood as regular variation of the finite-dimensional distributions. The jump sizes constitute a strictly stationary sequence. Using a continuous mapping argument acting on the point processes of the normalized jump sizes, we prove that the maximum increment of the random walk converges in distribution to a Fréchet distributed random variable.     

Quenched invariance principle for simple random walk on discrete point processes

We consider the simple random walk on random graphs generated by discrete point processes. This random walk moves on graphs whose vertex set is a random subset of a cubic lattice and whose edges are lines between any consecutive vertices on lines parallel to each coordinate axis. Under the assumption that the discrete point processes are finitely dependent and stationary, we prove that the quenched invariance principle holds, i.e., for almost every configuration of the point process, the path distribution of the walk converges weakly to that of a Brownian motion.     

Estimates for the Tail Probability of the Supremum of a Random Walk with Independent Increments

The authors investigate the tail probability of the supremum of a random walk with independent increments and obtain some equivalent assertions in the case that the increments are independent and identically distributed random variables with O-subexponential integrated distributions. A uniform upper bound is derived for the distribution of the supremum of a random walk with independent but non-identically distributed increments, whose tail distributions are dominated by a common tail distribution with an O-subexponential integrated distribution.     

Quantum random walk on the dual of SU (n)

Summary We study a quantum random walk onA(SU(n)), the von Neumann algebra of SU(n), obtained by tensoring the basic representation of SU(n). Two classical Markov chains are derived from this quantum random walk, by restriction to commutative subalgebras ofA(SU(n)), and the main result of the paper states that these two Markov chains are related by means of Doob'sh-processes.     

Random methods in 3-manifold theory

The surface map arising from a random walk on the mapping class group may be used as the gluing map for a Heegaard splitting, and the resulting 3-manifold is known as a random Heegaard splitting. We show that the splitting distance of random Heegaard splittings grows linearly in the length of the random walk, with an exponential decay estimate for the proportion with slower growth. We use this to obtain the limiting distribution of Casson invariants of random Heegaard splittings.     

Discrete approximation of symmetric jump processes on metric measure spaces

In this paper we give general criteria on tightness and weak convergence of discrete Markov chains to symmetric jump processes on metric measure spaces under mild conditions. As an application, we investigate discrete approximation for a large class of symmetric jump processes. We also discuss some application of our results to the scaling limit of random walk in random conductance.     

Shortest expected delay routing for Erlang servers

The queueing problem with Poisson arrivals and two identical parallel Erlang servers is analyzed for the case of shortest expected delay routing. This problem may be represented as a random walk on the integer grid in the first quadrant of the plane. An important aspect of the random walk is that it is possible to make large jumps in the direction of the boundaries. This feature gives rise to complicated boundary behavior. Generating function approaches to analyze this type of random walk seem to be extremely complicated and have not been successful yet. The approach presented in this paper directly solves the equilibrium equations. It is shown that the equilibrium distribution of the random walk can be written as an infinite linear combination of products. This linear combination is constructed in a compensation procedure. The starting solutions for this procedure are found by solving the shortest expected delay problem with instantaneous jockeying. The results can be used for an efficient computation of performance criteria, such as the waiting time distribution and the moments of the waiting time and the queue lengths.