Entropy Maximization for Markov and Semi-Markov Processes

The literature about maximum of entropy for Markov processes deals mainly with discrete-time Markov chains. Very few papers dealing with continuous-time jump Markov processes exist and none dealing with semi-Markov processes. It is the aim of this paper to contribute to fill this lack. We recall the basics concerning entropy for Markov and semi-Markov processes and we study several problems to give an overview of the possible directions of use of maximum entropy in connection with these processes. Numeric illustrations are presented, in particular in application to reliability.     

The Geometric Markov Renewal Processes with Application to Finance

We introduce the geometric Markov renewal processes as a model for a security market and study this processes in a series scheme. We consider its approximations in the form of averaged, merged and double averaged geometric Markov renewal processes. Weak convergence analysis and rates of convergence of ergodic geometric Markov renewal processes are presented. Martingale properties, infinitesimal operators of geometric Markov renewal processes are presented and a Markov renewal equation for expectation is derived. As an application, we consider the case of two ergodic classes. Moreover, we consider a generalized binomial model for a security market induced by a position dependent random map as a special case of a geometric Markov renewal process.     

非时齐马氏过程的耦合基本定理

本文将耦合方法应用于非时齐马氏过程,推广了时齐情形的耦合基本定理,为后续研究非时齐马氏过程的耦合提供了理论基础.     

Recent Results on Recurrent Solutions and Limit Distributions of SDEs

The limit distribution for homogeneous Markov processes is studied extensively and well understood, but it is not the case for inhomogeneous Markov processes. In this paper, we review some recent results on inhomogeneous Markov processes generated by non-autonomous stochastic (partial) differential equations (SDE in short). Under some suitable conditions, we show that the distribution of recurrent solutions of SDEs constitutes the limit distribution of the corresponding inhomogeneous Markov processes.     

非时齐马氏过程的随机单调性

本文研究了非时齐马氏过程的随机单调性问题.利用时齐的马氏过程随机单调性的相关证明方法,加以改进,获得了非时齐马氏过程随机单调性的显式判定方法,并进一步将这一充分性条件推广为等价条件.     

Birthing Markov processes at random rates

Summary This paper studies processes constructed by birthing the trajectories of a given Markov process along time according to random probabilities. Getoor has considered the case where the random probabilities are determined by comultiplicative functionals and proved for right processes that the post-birth process has the Markov property. Here randomizations of comultiplicative functionals are described which give rise to conditionally Markov processes. The main argument is developed for general Markov processes and the transition probabilities of the new process, including those from the pre-birth state, are explicited.     

非齐次可列马尔可夫过程的轨道性质

§1 状态分类 定义1.1 设I是非负整数集,P={P_(ij)(s,t)|i,j∈I,α≤s≤t≤b}是转移函数矩阵。称P对i在t右标准,若limp_(ii)(t,t+h)=1;称P对i在t左标准,若limP_(ii)(t-h,t)=1.若P对i在t同时为右标准的和左标准的,则称P对i在t标准。若P对i在每个t标准,则称P对i标准。P对i右标准或左标准与此类似。若P对每个i标准,则称P标准。P右标准或左标准与此类似(参看[5]、[6])。     

Hierarchic Markov processes and their applications in replacement models

In this paper a new notion of a hierarchic Markov process is introduced. It is a series of Markov decision processes called subprocesses built together in one Markov decision process called the main process. The hierarchic structure is specially designed to fit replacement models which in the traditional formulation as ordinary Markov decision processes are usually very large. The basic theory of hierarchic Markov processes is described and examples are given of applications in replacement models. The theory can be extended to fit a situation where the replacement decision depends on the quality of the new asset available for replacement.     

Isomorphism theorems for Markov chains

The isomorphism theorem of Dynkin is definitely an important tool to investigate the problems raised in terms of local times of Markov processes. This theorem concerns continuous time Markov processes. We give here an equivalent version for Markov chains.     

Strong Law of Large Numbers and Central Limit Theorems for Functionals of Inhomogeneous Semi-Markov Processes

Limit theorems for functionals of classical (homogeneous) Markov renewal and semi-Markov processes have been known for a long time, since the pioneering work of Pyke Schaufele (Limit theorems for Markov renewal processes, Ann. Math. Statist., 35(4):1746–1764, 1964). Since then, these processes, as well as their time-inhomogeneous generalizations, have found many applications, for example, in finance and insurance. Unfortunately, no limit theorems have been obtained for functionals of inhomogeneous Markov renewal and semi-Markov processes as of today, to the best of the authors’ knowledge. In this article, we provide strong law of large numbers and central limit theorem results for such processes. In particular, we make an important connection of our results with the theory of ergodicity of inhomogeneous Markov chains. Finally, we provide an application to risk processes used in insurance by considering a inhomogeneous semi-Markov version of the well-known continuous-time Markov chain model, widely used in the literature.     

Markov cubature rules for polynomial processes

We study discretizations of polynomial processes using finite state Markov processes satisfying suitable moment matching conditions. The states of these Markov processes together with their transition probabilities can be interpreted as Markov cubature rules. The polynomial property allows us to study such rules using algebraic techniques. Markov cubature rules aid the tractability of path-dependent tasks such as American option pricing in models where the underlying factors are polynomial processes.     

Strong Markov Properties for Markov Random Fields

Markov properties and strong Markov properties for random fields are defined and discussed. Special attention is given to those defined by I. V. Evstigneev. The strong Markov nature of Markov random fields with respect to random domains such as [0, L], where L is a multidimensional extension of a stopping time, is explored. A special case of this extension is shown to generalize a result of Merzbach and Nualart for point processes. As an additional example, Evstigneev's Markov and strong Markov properties are considered for independent increment jump processes.     

Markov dilation of Diffusion Type Processes and Its Application to the Financial Mathematics

The Markov dilation of diffusion type processes is defined. Infinitesimal operators and stochastic differential equations for the obtained Markov processes are described. Some applications to the integral representation for functionals of diffusion type processes and to the construction of a replicating portfolio for a non-terminal contingent claim are considered.     

TCP and iso-stationary transformations

We consider piecewise-deterministic Markov processes that occur as scaling limits of discrete-time Markov chains that describe the Transmission Control Protocol (TCP). The class of processes allows for general increase and decrease profiles. Our key observation is that stationary results for the general class follow directly from the stationary results for the idealized TCP process. The latter is a Markov process that increases linearly and experiences downward jumps at times governed by a Poisson process. To establish this connection, we apply space–time transformations that preserve the properties of the class of Markov processes.     

An invariance principle for non-symmetric Markov processes and reflecting diffusions in random domains

Summary We study an invariance principle for additive functionals of nonsymmetric Markov processes with singular mean forward velocities. We generalize results of Kipnis and Varadhan [KV] and De Masi et al. [De] in two directions: Markov processes are non-symmetric, and mean forward velocities are distributions. We study continuous time Markov processes. We use our result to homogenize non-symmetric reflecting diffusions in random domains.     

Trajectory composition of Poisson time changes and Markov counting systems

Changing time of simple continuous-time Markov counting processes by independent unit-rate Poisson processes results in Markov counting processes for which we provide closed-form transition rates via composition of trajectories and with which we construct novel, simpler infinitesimally over-dispersed processes.     

拟对称Markov链泛函型中心极限定理

本文研究了一般的马尔可夫链特别是拟对称马尔可夫链．利用Lyons-Meyer-Zheng对称马尔可夫过程的鞅分解，建立了泛函型中心极限定理．推广到了一般平稳遍历马尔可夫过程。     

A fluctuation theory for Markov chains

A fluctuation theory for Markov chains on an ordered countable state space is developed, using ladder processes. These are shown to be Markov renewal processes. Results are given for the joint distribution of the extremum (maximum or minimum) and the first time the extremum is achieved. Also a new classification of the states of a Markov chain is suggested. Two examples are given.     

Stochastic processes with orthogonal polynomial eigenfunctions

Markov processes which are reversible with either Gamma, Normal, Poisson or Negative Binomial stationary distributions in the Meixner class and have orthogonal polynomial eigenfunctions are characterized as being processes subordinated to well-known diffusion processes for the Gamma and Normal, and birth and death processes for the Poisson and Negative Binomial. A characterization of Markov processes with Beta stationary distributions and Jacobi polynomial eigenvalues is also discussed.     

Local asymptotic normality and mixed normality for Markov statistical models

Summary We prove local asymptotic normality (resp. local asymptotic mixed normality) of a statistical experiment, when the observation is a positive-recurrent (resp. null-recurrent, with an additional technical assumption) Markov chain or Markov step process, under rather mild regularity assumptions on the transition kernel for Markov chains, on the infinitesimal generator for Markov processes. The proof makes intensive use of Hellinger processes, thus avoiding almost completely to study the more complicated structure of the likelihoods themselves.