非齐次马氏决策过程的齐次化

该文考虑的是可数状态空间有限行动空间非齐次马氏决策过程的期望总报酬准则．与以往不同的是，我们是通过扩大状态空间的方法，将非齐次的马氏决策过程转化成齐次的马氏决策过程，于是非常简洁地得到了按传统的方法所得的主要结果．     

Self-Intersections of Markov Processes

We describe explicit conditions on the transition density functionsof a time-homogeneous continuous. Markov process so that almostevery path has multiple points. The application to two- andthree-dimensional diffusions is exhibited.     

The Weighted Transience and Recurrence of Markov Processes

Transience and recurrence are among the most important concepts in Markov processes. In this paper, we study the transience and recurrence for right processes with a given weight function, and characterize them by potentials, excessive functions, first hitting times and last exit times of the process. We also study the properties of recurrent states.     

马尔可夫过程的与时间有关的函数

设（Ｘｔ）是有转移函数的马尔可夫过程,其中Ｘｔ取值于状态空间（Ｅｔ,ξ,ｔ≥０。设ｆｔ是（Ｅｔ,ξ）到状态空间（Ｅｔ,ξ是的可测变换。本文给出了使（ｆｔ（Ｘｔ）仍是有转移函数的马尔可夫过程的充分条件,对于有函数的马尔可夫过程族,也讨论了类似的问题。     

The Tax Identity For Markov Additive Risk Processes

Taxed risk processes, i.e. processes which change their drift when reaching new maxima, represent a certain type of generalizations of Lévy and of Markov additive processes (MAP), since the times at which their Markovian mechanism changes are allowed to depend on the current position. In this paper we study generalizations of the tax identity of Albrecher and Hipp (2007) from the classical risk model to more general risk processes driven by spectrally-negative MAPs. We use the Sparre Andersen risk processes with phase-type interarrivals to illustrate the ideas in their simplest form.     

The Transformation Method for Continuous-Time Markov Decision Processes

In this paper, we show that a discounted continuous-time Markov decision process in Borel spaces with randomized history-dependent policies, arbitrarily unbounded transition rates and a non-negative reward rate is equivalent to a discrete-time Markov decision process. Based on a completely new proof, which does not involve Kolmogorov??s forward equation, it is shown that the value function for both models is given by the minimal non-negative solution to the same Bellman equation. A verifiable necessary and sufficient condition for the finiteness of this value function is given, which induces a new condition for the non-explosion of the underlying controlled process.     

非时齐马氏过程的耦合基本定理

本文将耦合方法应用于非时齐马氏过程,推广了时齐情形的耦合基本定理,为后续研究非时齐马氏过程的耦合提供了理论基础.     

Markov Processes with Equal Capacities

Let X and be transient standard Markov processes in weak duality with respect to a -finite measure m. Let (Y, , ) be a second dual pair with the same state space E as (X, , m). Let Cap ^X and Cap ^Y be the 0-order capacities associated with (X, , m) and (Y, , ), and let V and denote the potential kernels for Y and . Assume that singletons are polar with respect to both X and Y, and that semipolar sets are of capacity zero for both dual pairs. We show that if Cap ^X (B)=Cap ^Y (B) for every Borel subset of E then there is a strictly increasing continuous additive functional D=(D _t) _t0 of (X, , m) such that

On weakly mixing Markov Processes

A result of England and Martin on weak mixing (see [6]) is extended to Markov Processes in a strengthened form, and also to continuous time Markov Processes.     

Lumpability and Commutativity of Markov Processes

Abstract

We introduce the concepts of lumpability and commutativity of a continuous time discrete state space Markov process, and provide a necessary and sufficient condition for a lumpable Markov process to be commutative. Under suitable conditions we recover some of the basic quantities of the original Markov process from the jump chain of the lumped Markov process.     

马尔可夫骨架过程的极限分布

本文运用基本更新定理和Smith关键更新定理等理论和方法,对马尔可夫骨架过程的极限分布进行深入研究,得到主要结果如下:去掉了原有结果中要求的绝对连续的条件,给出了马尔可夫骨架过程极限分布存在的充分条件;得到了马尔可夫骨架过程极限分布的具体公式,并证明了该极限分布为概率分布.     

The Geometric Markov Renewal Processes with Application to Finance

We introduce the geometric Markov renewal processes as a model for a security market and study this processes in a series scheme. We consider its approximations in the form of averaged, merged and double averaged geometric Markov renewal processes. Weak convergence analysis and rates of convergence of ergodic geometric Markov renewal processes are presented. Martingale properties, infinitesimal operators of geometric Markov renewal processes are presented and a Markov renewal equation for expectation is derived. As an application, we consider the case of two ergodic classes. Moreover, we consider a generalized binomial model for a security market induced by a position dependent random map as a special case of a geometric Markov renewal process.     

集值马尔可夫过程的定义及其相关问题

本文从集值马尔可夫过程的最简单定义出发,讨论了它的一系列等价命题,证明了它与已有的两种定义的等价性,并就集值随机过程与其数值特征过程的马尔可夫性的关系进行了讨论,完善了已有的结果。     

Directed Markov Point Processes as Limits of Partially Ordered Markov Models

In this paper, we consider spatial point processes and investigate members of a subclass of the Markov point processes, termed the directed Markov point processes (DMPPs), whose joint distribution can be written in closed form and, as a consequence, its parameters can be estimated directly. Furthermore, we show how the DMPPs can be simulated rapidly using a one-pass algorithm. A subclass of Markov random fields on a finite lattice, called partially ordered Markov models (POMMs), has analogous structure to that of DMPPs. In this paper, we show that DMPPs are the limits of auto-Poisson and auto-logistic POMMs. These and other results reveal a close link between inference and simulation for DMPPs and POMMs.     

Gilt-Edged Switching and Markov Decision Processes

A model for switching between gilt-edged securities is developed using a modified version of Howard's Markov decision algorithm. The model makes use of empirical observations of the behaviour of relative price movements. It produced some interesting results in the theory of Markov decision processes, and empirical tests of methods of implementation, which allow for constraints not included in the formal model, showed very promising results.     

Measure-Valued Markov Processes and Stochastic Flows

We consider a new class of Markov processes in the space of measures with constant mass. We present the construction of such processes in terms of probabilities that control the motion of individual particles. We study additive functionals of such processes and give examples related to stochastic flows with interaction.     

Markov Processes Associated to Resistance Forms

We give a method to associate a right Markov process to a resistance form, that is different from the one based on the regularity assumption for the associated Dirichlet form. We use tools from potential theory associated to resolvents of kernels and cones of potentials (H-cones). The induced capacity and processes obtained by time change with the inverses of continuous additive functionals also occur.