Diffusion processes with delay at the endpoints of a segment

A continuous semi-Markov process with a segment as the range of values is considered. This process coincides with a diffusion process inside the segment, i.e., up to the first hitting time of the boundary of the segment and at any time when the process leaves the boundary. The class of such processes consists of Markov processes with reflection at the boundaries (instantaneously or with a delay) and semi-Markov processes with intervals of constancy on some boundary. We derive conditions of existence of such a process in terms of a semi-Markov transition generating function on the boundary. The method of imbedded alternating renewal processes is applied to find a stationary distribution of the process. Bibliography: 3 titles. __________ Translated from Zapiski Nauchnykh Seminarov POMI, Vol. 351, 2007, pp. 284–297.     

Stochastic Processes with Age-Dependent Transition Rates

We study stochastic processes with age-dependent transition rates. A typical example of such a process is a semi-Markov process which is completely determined by the holding time distributions in each state and the transition probabilities of the embedded Markov chain. The process we construct generalizes semi-Markov processes. One important feature of this process is that unlike semi-Markov processes the transition probabilities of this process are age-dependent. Under certain condition we establish the Feller property of the process. Finally, we compute the limiting distribution of the process.     

半马尔可夫过程的极限分布及其推广

本文运用马尔可夫骨架过程的极限理论研究齐次可列半马尔可夫过程,得到其极限分布.当更新间隔的分布不是格子分布时,本文的结果和邓永录等[1]中的结果一致,但采用的方法不同,本文采用的是马尔可夫骨架过程的理论方法,而[1]中采用的是交替更新过程的方法;而且关于更新间隔服从格子分布的情形,[1]中没有研究,而本文给出了结果.最...     

Computational methods for discrete hidden semi-Markov chains

We propose a computational approach for implementing discrete hidden semi-Markov chains. A discrete hidden semi-Markov chain is composed of a non-observable or hidden process which is a finite semi-Markov chain and a discrete observable process. Hidden semi-Markov chains possess both the flexibility of hidden Markov chains for approximating complex probability distributions and the flexibility of semi-Markov chains for representing temporal structures. Efficient algorithms for computing characteristic distributions organized according to the intensity, interval and counting points of view are described. The proposed computational approach in conjunction with statistical inference algorithms previously proposed makes discrete hidden semi-Markov chains a powerful model for the analysis of samples of non-stationary discrete sequences. Copyright © 1999 John Wiley & Sons, Ltd.     

On Some Consequences of the Equation for the Markov Renewal Function of a Semi-Markov Process

We obtain chains of equations that relate the sojourn times of a semi-Markov process in a set of states to its Markov renewal function. We use the mathematical apparatus of the theory of Markov and semi-Markov processes.__________Translated from Ukrains’kyi Matematychnyi Zhurnal, Vol. 56, No. 12, pp. 1684 – 1690, December, 2004.     

Semi-Markov processes and α-invariant distributions

A semi-Markov process is easily made Markov by adding some auxiliary random variables. This paper discusses the I-type quasi-stationary distributions of such “extended” processes, and the α-invariant distributions for the corresponding Markov transition probabilities; and we show that there is an intimate relation between the two. The results have relevance in the study of the time to “absorption” or “death” of semi-Markov processes. The particular case of a terminating renewal process is studied as an example.     

半马氏环境连续时间马氏决策过程：平均准则

本文讨论半马氏环境连续时间马氏决策过程中的平均准则．首先讨论了半马氏报酬过程中的逼近问题，进而讨论平均目标函数逼近问题。     

Entropy Maximization for Markov and Semi-Markov Processes

The literature about maximum of entropy for Markov processes deals mainly with discrete-time Markov chains. Very few papers dealing with continuous-time jump Markov processes exist and none dealing with semi-Markov processes. It is the aim of this paper to contribute to fill this lack. We recall the basics concerning entropy for Markov and semi-Markov processes and we study several problems to give an overview of the possible directions of use of maximum entropy in connection with these processes. Numeric illustrations are presented, in particular in application to reliability.     

A semi-Markov approach for modelling asset deterioration

Considerable benefits have been gained from using Markov decision processes to select condition-based maintenance policies for the asset management of infrastructure systems. A key part of the method is using a Markov process to model the deterioration of condition. However, the Markov model assumes constant transition probabilities irrespective of how long an item has been in a state. The semi-Markov model relaxes this assumption. This paper describes how to fit a semi-Markov model to observed condition data and the results achieved on two data sets. Good results were obtained even where there was only 1 year of observation data.     

Non-Stationary Semi-Markov Decision Processes on a Finite Horizon

We introduce and study a class of non-stationary semi-Markov decision processes on a finite horizon. By constructing an equivalent Markov decision process, we establish the existence of a piecewise open loop relaxed control which is optimal for the finite horizon problem.     

Stochastic Systems with Averaging in the Scheme of Diffusion Approximation

We propose a system approach to the asymptotic analysis of stochastic systems in the scheme of series with averaging and diffusion approximation. Stochastic systems are defined by Markov processes with locally independent increments in a Euclidean space with random switchings that are described by jump Markov and semi-Markov processes. We use the asymptotic analysis of Markov and semi-Markov random evolutions. We construct the diffusion approximation using the asymptotic decomposition of generating operators and solutions of problems of singular perturbation for reducibly inverse operators. __________ Translated from Ukrains'kyi Matematychnyi Zhurnal, Vol. 57, No. 9, pp. 1235–1252, September, 2005.     

Sufficient Stochastic Maximum Principle for the Optimal Control of Semi-Markov Modulated Jump-Diffusion with Application to Financial Optimization

The finite state semi-Markov process is a generalization over the Markov chain in which the sojourn time distribution is any general distribution. In this article, we provide a sufficient stochastic maximum principle for the optimal control of a semi-Markov modulated jump-diffusion process in which the drift, diffusion, and the jump kernel of the jump-diffusion process is modulated by a semi-Markov process. We also connect the sufficient stochastic maximum principle with the dynamic programming equation. We apply our results to finite horizon risk-sensitive control portfolio optimization problem and to a quadratic loss minimization problem.     

Estimating Hidden Semi-Markov Chains From Discrete Sequences

This article addresses the estimation of hidden semi-Markov chains from nonstationary discrete sequences. Hidden semi-Markov chains are particularly useful to model the succession of homogeneous zones or segments along sequences. A discrete hidden semi-Markov chain is composed of a nonobservable state process, which is a semi-Markov chain, and a discrete output process. Hidden semi-Markov chains generalize hidden Markov chains and enable the modeling of various durational structures. From an algorithmic point of view, a new forward-backward algorithm is proposed whose complexity is similar to that of the Viterbi algorithm in terms of sequence length (quadratic in the worst case in time and linear in space). This opens the way to the maximum likelihood estimation of hidden semi-Markov chains from long sequences. This statistical modeling approach is illustrated by the analysis of branching and flowering patterns in plants.     

Markov modulation of a two-sided reflected Brownian motion with application to fluid queues

In this paper, we study a reflected Markov-modulated Brownian motion with a two sided reflection in which the drift, diffusion coefficient and the two boundaries are (jointly) modulated by a finite state space irreducible continuous time Markov chain. The goal is to compute the stationary distribution of this Markov process, which in addition to the complication of having a stochastic boundary can also include jumps at state change epochs of the underlying Markov chain because of the boundary changes. We give the general theory and then specialize to the case where the underlying Markov chain has two states.     

Initial and Final Backward and Forward Discrete Time Non-homogeneous Semi-Markov Credit Risk Models

In this paper we show how it is possible to construct an efficient Migration models in the study of credit risk problems presented in Jarrow et al. (Rev Financ Stud 10:481–523, 1997) with Markov environment. Recently it was introduced the semi-Markov process in the migration models (D’Amico et al. Decis Econ Finan 28:79–93, 2005a). The introduction of semi-Markov processes permits to overtake some of the Markov constraints given by the dependence of transition probabilities on the duration into a rating category. In this paper, it is shown how it is possible to take into account simultaneously backward and forward processes at beginning and at the end of the time in which the credit risk model is observed. With such a generalization, it is possible to consider what happens inside the time after the first transition and before the last transition where the problem is studied. This paper generalizes other papers presented before. The model is presented in a discrete time environment.     

Discrete type shock semi-markov decision processes with borel state space

This paper investigates discrete type shock semi-Markov decision processes (SMDP for short) with Borel state and action space. The discrete type shock SMDP describes a system which behaves like a discrete type SMDP, except that the system is subject to random shocks from its environment. Following each shock, an instantaneous state transition occurs and the parameters of the SMDP are changed. After presenting the model, we transform the discrete type shock SMDP into an equivalent discrete time Markov decision process under the condition that one of the assumptions P, N, D, holds. So the most results from discrete time Markov decision processes can be generalized directly to hold for the discrete type shock SMDP.     

Number of deaths in semi-Markov compartmental system with branching particles

A semi-Markov compartmental system in which the particles reproduce similar particles, according to a Markov branching process, is considered. Asymptotic behavior of the mean matrix of the number of particles that die during the time interval (0, t) is discussed. Explicit expressions are obtained in some special cases.     

Convergence of a semi-Markov process and an accompanying

We propose an approach to the proof of the weak convergence of a semi-Markov process to a Markov process under certain conditions imposed on local characteristics of the semi-Markov process.     

Threshold dynamics and ergodicity of an SIRS epidemic model with semi-Markov switching

This work studies the threshold dynamics and ergodicity of a stochastic SIRS epidemic model with the disease transmission rate driven by a semi-Markov process. The semi-Markov process used in this paper for describing a randomly changing environment is a very large extension of the most common Markov regime-switching process. We define a basic reproduction number for the semi-Markov regime-switching environment and show that its position with respect to 1 determines the extinction or persistence of the disease. In the case of disease persistence, we give mild sufficient conditions for ensuring the existence and absolute continuity of the invariant probability measure. Under the same conditions, we also prove the global attractivity of the Ω-limit set of the system and the convergence in total variation norm of the transition probability to the invariant measure. Compared with the existing results in the Markov regime-switching environment, the results generalized require almost no additional conditions.     

Storage processes in the Poisson approximation scheme

Discrete storage processes defined by sums of random variables on a Markov or a semi-Markov process are approximated by compound Poisson processes with continuous drift on increasing time intervals.