Gauge theorems for resolvents with application to Markov processes

Summary LetV=(V )0 be a (not necessarily sub-Markovian) resolvent such that the kernelV for some 0 is compact and irreducible. We prove the following general gauge theorem: If there exists at least oneV-excessive function which is notV-inviriant, thenV ₀ is bounded.This result will be applied to resolventsU ^M arising from perturbation of sub-Markovian right resolventsU by multiplicative functionalsM (not necessarily supermartingale), for instance, by Feynman-Kac functionals. Among others, this leads to an extension of the gauge theorem of Chung/Rao and even of one direction of the conditional gauge theorem of Falkner and Zhao.     

Local limit theorems on the convergence of Markov chains to diffusion processes

One proves the uniform convergence of the densities of the finite-dimensional distributions of certain families of Markov chains to the densities of the finite-dimensional distributions of a nondegenerate diffusion process.Translated from Veroyatnostnye Raspredeleniya i Matematicheskaya Statistika, pp. 277–292, 1986.The author is grateful to S. A. Molchanov for numerous useful discussions.     

An overshoot approach to recurrence and transience of Markov processes

We develop criteria for recurrence and transience of one-dimensional Markov processes which have jumps and oscillate between +∞ and −∞. The conditions are based on a Markov chain which only consists of jumps (overshoots) of the process into complementary parts of the state space.In particular, we show that a stable-like process with generator −(−Δ)^α(x)/2 such that α(x)=α for x<−R and α(x)=β for x>R for some R>0 and α,β∈(0,2) is transient if and only if α+β<2, otherwise it is recurrent.As a special case, this yields a new proof for the recurrence, point recurrence and transience of symmetric α-stable processes.     

An envelope theorem and some applications to discounted Markov decision processes

In this paper, an Envelope Theorem (ET) will be established for optimization problems on Euclidean spaces. In general, the Envelope Theorems permit analyzing an optimization problem and giving the solution by means of differentiability techniques. The ET will be presented in two versions. One of them uses concavity assumptions, whereas the other one does not require such kind of assumptions. Thereafter, the ET established will be applied to the Markov Decision Processes (MDPs) on Euclidean spaces, discounted and with infinite horizon. As the first application, several examples (including some economic models) of discounted MDPs for which the et allows to determine the value iteration functions will be presented. This will permit to obtain the corresponding optimal value functions and the optimal policies. As the second application of the ET, it will be proved that under differentiability conditions in the transition law, in the reward function, and the noise of the system, the value function and the optimal policy of the problem are differentiable with respect to the state of the system. Besides, various examples to illustrate these differentiability conditions will be provided. This work was partially supported by Benemérita Universidad Aut ónoma de Puebla (BUAP) under grant VIEP-BUAP 38/EXC/06-G, by Consejo Nacional de Ciencia y Tecnología (CONACYT), and by Evaluation-orientation de la COopération Scientifique (ECOS) under grant CONACyT-ECOS M06-M01.     

Bivariate Markov chains converging to Lamperti transform Markov additive processes

Motivated by various applications, we describe the scaling limits of bivariate Markov chains $(X,J)$ on ${\mathbb{Z}}_{+}\times \{1,\dots ,\kappa \}$ where $X$ can be viewed as a position marginal and $\{1,\dots ,\kappa \}$ is a set of $\kappa$ types. The chain starts from an initial value $(n,i)\in {\mathbb{Z}}_{+}\times \{1,\dots ,\kappa \}$, with $i$ fixed and $n\to \infty$, and typically we will assume that the macroscopic jumps of the marginal $X$ are rare, i.e. arrive with a probability proportional to a negative power of the current state. We also assume that $X$ is non-increasing. We then observe different asymptotic regimes according to whether the rate of type change is proportional to, faster than, or slower than the macroscopic jump rate. In these different situations, we obtain in the scaling limit Lamperti transforms of Markov additive processes, that sometimes reduce to standard positive self-similar Markov processes. As first examples of applications, we study the number of collisions in coalescents in varying environment and the scaling limits of Markov random walks with a barrier. This completes previous results obtained by Haas and Miermont (2011) and Bertoin and Kortchemski (2016) in the monotype setting. In a companion paper, we will use these results as a building block to study the scaling limits of multi-type Markov branching trees, with applications to growing models of random trees and multi-type Galton–Watson trees.     

Nonhomogeneous Markov processes

We present the foundations of the theory of nonhomogeneous Markov processes in general state spaces and we give a survey of the fundamental papers in this topic. We consider the following questions:

1. The existence of transition functions for a Markov process.
2. The construction of regularization of processes.
3. The properties of right and left processes: the strict Markov property, the behavior of excessive functions, etc.
4. The relation of right and left processes with dual homogeneous processes and the application of the results of the nonhomogeneous theory to dual homogeneous processes, etc.

     

Corner Markov processes

In this work, a corner Markov type of property is suggested as a multiparameter interpretation of the classical Markov property. The first result presents one-dimensional conditions on a random field { _z },z ₊ ² , which, together with a Cairoli-WalshF4-type of condition, are sufficient for this corner Markov property. In the second part of this work two approaches to relevant strong Markov properties are discussed.     

Aggregation of Markov processes: Axiomatization

We give two simple axioms that characterize a simple functional form for aggregation of column stochastic matrices (i.e., Markov processes). Several additional observations are made about such aggregation, including the special case in which the aggregated process is Markovian relative to the original one.     

A note on stochastic order of probability measures and an application to Markov processes

Summary In this paper we prove under a general condition on the relation between metric and partial order of the underlying space an equivalence to stochastic order of probability measures. Secondly we show that the set of all probability measures which are placed stochastically between two fixed probability measures is compact in the topology of weak convergence. Using weak-convergence arguments we apply these two theorems to Markov processes, getting results on bounds and existence of their invariant probability measures. Finally, we give a transience criterion for some semi-Markov processes.

---

Zusammenfassung Zur stochastischen Vergleichbarkeit von Wahrscheinlichkeitsmaßen wird unter einer allgemeinen Voraussetzung an die Beziehung zwischen Metrik und Halbordnung des zugrundeliegenden Raumes eine äquivalente Formulierung bewiesen. Außerdem wird gezeigt, daß die Menge aller Wahrscheinlichkeitsmaße, die bezüglich der stochastischen Ordnung zwischen zwei festen Wahrscheinlichkeitsmaßen liegen, kompakt in der Topologie der schwachen Konvergenz ist. Beide Ergebnisse liefern mittels schwacher Konvergenz Schranken und Existenz von invarianten Wahrscheinlichkeitsmaßen Markoffscher Prozesse. Eine Anwendung, die ein Transienzkriterium für gewisse semi-Markoffsche Prozesse ergibt, schließt sich an.

     

A fundamental property of Markov processes with an application to equivalence under time changes

It is shown that Markov processes traverse their trajectories in just one way, and applications are given to the Blumenthal, Getoor and McKean theorem. This paper is dedicated to the memory of Shlomo Horowitz The second author received partial support from the Canadian Research Council and National Science Foundation (MCS-7606048).     

Constrictive Markov operators induced by Markov processes

Consider a constrictive Markov operator \(T:L^1(X, \Sigma , \mu ) \rightarrow L^1(X, \Sigma , \mu )\) defined on a finite measure space \((X, \Sigma , \mu )\). We give a necessary and sufficient condition for a constrictive Markov operator T which is an integral operator with stochastic kernel satisfying \(T\mathbf {1}_X=\mathbf {1}_X\).