On Markov diffusion processes with delayed reflection from boundaries of a segment

A continuous semi-Markov process with values in a closed interval is considered. This process coincides with a Markov diffusion process inside the interval. Thus, violation of the Markov property is only possible at the boundary of the interval. We prove a sufficient condition under which a semi-Markov process is Markov. We show that, in addition to Markov processes with instantaneous reflection from the boundary of the interval. there exists a class of Markov processes with delayed reflection from the boundary. Such a process has a positive average measure of time at which its trajectory belongs to the boundaries. This gives a different proof of a similar result by Gikhman and Skorokhod of 1968. Bibliography: 5 titles.     

The semi-Markov beta-Stacy process: a Bayesian non-parametric prior for semi-Markov processes.

The literature on Bayesian methods for the analysis of discrete-time semi-Markov processes is sparse. In this paper, we introduce the semi-Markov beta-Stacy process, a stochastic process useful for the Bayesian non-parametric analysis of semi-Markov processes. The semi-Markov beta-Stacy process is conjugate with respect to data generated by a semi-Markov process, a property which makes it easy to obtain probabilistic forecasts. Its predictive distributions are characterized by a reinforced random walk on a system of urns.

     

Stochastic Processes with Age-Dependent Transition Rates

We study stochastic processes with age-dependent transition rates. A typical example of such a process is a semi-Markov process which is completely determined by the holding time distributions in each state and the transition probabilities of the embedded Markov chain. The process we construct generalizes semi-Markov processes. One important feature of this process is that unlike semi-Markov processes the transition probabilities of this process are age-dependent. Under certain condition we establish the Feller property of the process. Finally, we compute the limiting distribution of the process.     

A Non-Homogeneous Continuous Time Semi-Markov Model for the Study of Accumulated Claim Process

The accumulated claim process is the summed total of all claims starting from time t. The semi-Markov environment, at authors’ opinion, is able to follow the evolution of this process. In the paper a continuous time non-homogeneous semi-Markov model with a denumerable set of states will be used to follow the stochastic evolution of the accumulated claim process.     

Semi-Markov processes and α-invariant distributions

A semi-Markov process is easily made Markov by adding some auxiliary random variables. This paper discusses the I-type quasi-stationary distributions of such “extended” processes, and the α-invariant distributions for the corresponding Markov transition probabilities; and we show that there is an intimate relation between the two. The results have relevance in the study of the time to “absorption” or “death” of semi-Markov processes. The particular case of a terminating renewal process is studied as an example.     

Semi-Markov modulated Poisson process: probabilistic and statistical analysis

We consider a Poisson process that is modulated in such a way that the arrival rate at any time depends on the state of a semi-Markov process. This presents an interesting generalization of Poisson processes with important implications in real life applications. Our analysis concentrates on the transient as well as the long term behaviour of the arrival count and the arrival time processes. We discuss probabilistic as well as statistical issues related to various quantities of interest.     

Nonparametric Estimation for Semi-Markov Processes Based on its Hazard Rate Functions

The problem of estimating the Markov renewal matrix and the semi-Markov transition matrix based on a history of a finite semi-Markov process censored at time T (fixed) is addressed for the first time. Their asymptotic properties are studied. We begin by the definition of the transition rate of this process and propose a maximum likelihood estimator for the hazard rate functions and then we show that this estimator is uniformly strongly consistent and converges weakly to a normal random variable. We construct a new estimator for an absolute continous semi-Markov kernel and give detailed derivation of uniform strong consistency and weak convergence of this estimator as the censored time tends to infinity. This revised version was published online in June 2006 with corrections to the Cover Date.     

Strong Law of Large Numbers and Central Limit Theorems for Functionals of Inhomogeneous Semi-Markov Processes

Limit theorems for functionals of classical (homogeneous) Markov renewal and semi-Markov processes have been known for a long time, since the pioneering work of Pyke Schaufele (Limit theorems for Markov renewal processes, Ann. Math. Statist., 35(4):1746–1764, 1964). Since then, these processes, as well as their time-inhomogeneous generalizations, have found many applications, for example, in finance and insurance. Unfortunately, no limit theorems have been obtained for functionals of inhomogeneous Markov renewal and semi-Markov processes as of today, to the best of the authors’ knowledge. In this article, we provide strong law of large numbers and central limit theorem results for such processes. In particular, we make an important connection of our results with the theory of ergodicity of inhomogeneous Markov chains. Finally, we provide an application to risk processes used in insurance by considering a inhomogeneous semi-Markov version of the well-known continuous-time Markov chain model, widely used in the literature.     

Initial and Final Backward and Forward Discrete Time Non-homogeneous Semi-Markov Credit Risk Models

In this paper we show how it is possible to construct an efficient Migration models in the study of credit risk problems presented in Jarrow et al. (Rev Financ Stud 10:481–523, 1997) with Markov environment. Recently it was introduced the semi-Markov process in the migration models (D’Amico et al. Decis Econ Finan 28:79–93, 2005a). The introduction of semi-Markov processes permits to overtake some of the Markov constraints given by the dependence of transition probabilities on the duration into a rating category. In this paper, it is shown how it is possible to take into account simultaneously backward and forward processes at beginning and at the end of the time in which the credit risk model is observed. With such a generalization, it is possible to consider what happens inside the time after the first transition and before the last transition where the problem is studied. This paper generalizes other papers presented before. The model is presented in a discrete time environment.     

Stochastic integral with respect to a semi-Markov process of diffusion type

We consider a multidimensional semi-Markov process of diffusion type. A stochastic integral with respect to the semi-Markov process is defined in terms of asymptotics related to the first exit time from a small neighborhood of the starting point of the process, and, in particular, in terms of its characteristic operator. This integral is equal to the sum of two other integrals: the first one is a curvilinear integral with respect to an additive functional defined in terms of the expected first exit time from a small neighborhood, and the second one is a stochastic integral with respect to a martingale of special kind. To prove the existence and to derive the properties of the integral, both the method of deducing sequences and that of inscribed ellipsoids are used. For Markov processes of diffusion type, the new definition of the stochastic integral is reduced to the standard one. Bibliography: 8 titles. __________ Translated from Zapiski Nauchnykh Seminarov POMI, Vol. 328, 2005, pp. 251–276.     

Stationary distribution of a process of random semi-Markov evolution with delaying screens in the case of balance

We determine a stationary measure for a process defined by a differential equation with phase space on the segment [V ₀, V ₁] and constant values of a vector field that depend on a controlling semi-Markov process with finite set of states. __________ Translated from Ukrains’kyi Matematychnyi Zhurnal, Vol. 58, No. 3, pp. 381–387, March, 2006.     

Analysis of a single server queue with semi-Markovian service interruption

In this paper we analyze a discrete-time single server queue where the service time equals one slot. The numbers of arrivals in each slot are assumed to be independent and identically distributed random variables. The service process is interrupted by a semi-Markov process, namely in certain states the server is available for service while the server is not available in other states. We analyze both the transient and steady-state models. We study the generating function of the joint probability of queue length, the state and the residual sojourn time of the semi-Markov process. We derive a system of Hilbert boundary value problems for the generating functions. The system of Hilbert boundary value problems is converted to a system of Fredholm integral equations. We show that the system of Fredholm integral equations has a unique solution. This revised version was published online in June 2006 with corrections to the Cover Date.     

Absolute continuity of measures in the class of Markov and semi-Markov processes of diffusion type

The property of absolute continuity of measures in the class of semi-Markov processes of diffusion type is investigated. The measure of such a process can be represented in the form of a composition of two measures. The first one is the distribution of a random track, and the second one is a conditional distribution of the time run along the track. The desired density (if it exists) is represented in the form of the product of the corresponding two densities. The first density is based on the asymptotic of the distribution density of the first exit point for the process exiting from an ellipsoidal neighborhood of its initial point. In terms of the associated Markov process and the induced Wiener process, this formula coincides with the known formula for the density of a diffusion-type Markov process measure. The second density is based on the semi-Markov property, which implies that the conditional distribution of the time run along a given track is the distribution of a monotone process with independent increments. Bibliography: 6 titles.Translated from Zapiski Nauchnykh Seminarov POMI, Vol. 294, 2002, pp. 216–244.This research was supported by the Russian Foundation for Basic Research, grant 01-01-00613, and by the Program Leading Scientific Schools, grant 00-15-96019.Translated by B. P. Harlamov.     

First passage models for denumerable semi-Markov decision processes with nonnegative discounted costs

This paper considers a first passage model for discounted semi-Markov decision processes with denumerable states and nonnegative costs.The criterion to be optimized is the expected discounted cost incurred during a first passage time to a given target set.We first construct a semi-Markov decision process under a given semi-Markov decision kernel and a policy.Then,we prove that the value function satisfies the optimality equation and there exists an optimal(or e-optimal) stationary policy under suitable conditions by using a minimum nonnegative solution approach.Further we give some properties of optimal policies.In addition,a value iteration algorithm for computing the value function and optimal policies is developed and an example is given.Finally,it is showed that our model is an extension of the first passage models for both discrete-time and continuous-time Markov decision processes.     

Storage processes in the Poisson approximation scheme

Discrete storage processes defined by sums of random variables on a Markov or a semi-Markov process are approximated by compound Poisson processes with continuous drift on increasing time intervals.     

Insensitivity of Generalized Semi-Markov Processes Evolving in a Random Environment

The theory of insensitivity within generalized semi-Markov processes is extended to cover the case where such a process evolves in a random environment; that is, when the decay rates and transition probabilities are functions of the state of an extraneous environmental process.     

A Stagewise Action Elimination Algorithm for the Discounted Semi-Markov Problem

An efficient algorithm for solving discounted semi-Markov (Markov-renewal) problems is proposed. The value iteration method of dynamic programming is used in conjunction with a test for non-optimal actions. A non-optimality test for the discounted semi-Markov processes, which is an extension of Hastings and Van Nunens (1976) test for the undiscounted or discounted returns with infinite or finite planning horizon, is used to identify actions which cannot be optimal at the current stage of a discounted semi-Markov process. The test proposed eliminates actions for one or more stages after which they may enter the set of possibly optimal actions, but such re-entries cease as convergence proceeds.     

Absolute Continuity of Measures in the Class of Markov and Semi-Markov Processes of Diffusion Type

The property of absolute continuity of measures in the class of one-dimensional semi-Markov processes of diffusion type is investigated. The measure of such a process can be composed of two measures. The first one is a distribution of a random track, and the second one is a conditional distribution of a time run along the track. The desired density is represented in the form of product of two corresponding densities.