Some Reward Paths in Semi-Markov Models with Stochastic Selection of the Transition Probabilities

In the present paper, the reward paths in non homogeneous semi-Markov systems in discrete time are examined with stochastic selection of the transition probabilities. The mean entrance probabilities and the mean rewards in the course of time are evaluated. Then the rate of the total reward for the homogeneous case is examined and the mean total reward is evaluated by means of p.g.f’s.      

Stochastic Processes with Age-Dependent Transition Rates

We study stochastic processes with age-dependent transition rates. A typical example of such a process is a semi-Markov process which is completely determined by the holding time distributions in each state and the transition probabilities of the embedded Markov chain. The process we construct generalizes semi-Markov processes. One important feature of this process is that unlike semi-Markov processes the transition probabilities of this process are age-dependent. Under certain condition we establish the Feller property of the process. Finally, we compute the limiting distribution of the process.     

Numerical Approach for Assessing System Dynamic Availability Via Continuous Time Homogeneous Semi-Markov Processes

Continuous-time homogeneous semi-Markov processes (CTHSMP) are important stochastic tools to model reliability measures for systems whose future behavior is dependent on the current and next states occupied by the process as well as on sojourn times in these states. A method to solve the interval transition probabilities of CTHSMP consists of directly applying any general quadrature method to the N ² coupled integral equations which describe the future behavior of a CTHSMP, where N is the number of states. However, the major drawback of this approach is its considerable computational effort. In this work, it is proposed a new more efficient numerical approach for CTHSMPs described through either transition probabilities or transition rates. Rather than N ² coupled integral equations, the approach consists of solving only N coupled integral equations and N straightforward integrations. Two examples in the context of availability assessment are presented in order to validate the effectiveness of this method against the comparison with the results provided by the classical and Monte Carlo approaches. From these examples, it is shown that the proposed approach is significantly less time-consuming and has accuracy comparable to the method of N ² computational effort.     

A semi-Markov approach for modelling asset deterioration

Considerable benefits have been gained from using Markov decision processes to select condition-based maintenance policies for the asset management of infrastructure systems. A key part of the method is using a Markov process to model the deterioration of condition. However, the Markov model assumes constant transition probabilities irrespective of how long an item has been in a state. The semi-Markov model relaxes this assumption. This paper describes how to fit a semi-Markov model to observed condition data and the results achieved on two data sets. Good results were obtained even where there was only 1 year of observation data.     

Semi-Markov processes and α-invariant distributions

A semi-Markov process is easily made Markov by adding some auxiliary random variables. This paper discusses the I-type quasi-stationary distributions of such “extended” processes, and the α-invariant distributions for the corresponding Markov transition probabilities; and we show that there is an intimate relation between the two. The results have relevance in the study of the time to “absorption” or “death” of semi-Markov processes. The particular case of a terminating renewal process is studied as an example.     

Insensitivity of Generalized Semi-Markov Processes Evolving in a Random Environment

The theory of insensitivity within generalized semi-Markov processes is extended to cover the case where such a process evolves in a random environment; that is, when the decay rates and transition probabilities are functions of the state of an extraneous environmental process.     

Exchangeably weighted bootstraps of empirical estimators of a semi-Markov kernel

A general notion of bootstrapped empirical estimators, of the semi-Markov kernels and of the conditional transition probabilities for semi-Markov processes with countable state space, constructed by exchangeably weighting sample, is introduced. Asymptotic properties of these generalized bootstrapped empirical distributions are obtained by means of the martingale approach.     

Estimating Hidden Semi-Markov Chains From Discrete Sequences

This article addresses the estimation of hidden semi-Markov chains from nonstationary discrete sequences. Hidden semi-Markov chains are particularly useful to model the succession of homogeneous zones or segments along sequences. A discrete hidden semi-Markov chain is composed of a nonobservable state process, which is a semi-Markov chain, and a discrete output process. Hidden semi-Markov chains generalize hidden Markov chains and enable the modeling of various durational structures. From an algorithmic point of view, a new forward-backward algorithm is proposed whose complexity is similar to that of the Viterbi algorithm in terms of sequence length (quadratic in the worst case in time and linear in space). This opens the way to the maximum likelihood estimation of hidden semi-Markov chains from long sequences. This statistical modeling approach is illustrated by the analysis of branching and flowering patterns in plants.     

Computational methods for discrete hidden semi-Markov chains

We propose a computational approach for implementing discrete hidden semi-Markov chains. A discrete hidden semi-Markov chain is composed of a non-observable or hidden process which is a finite semi-Markov chain and a discrete observable process. Hidden semi-Markov chains possess both the flexibility of hidden Markov chains for approximating complex probability distributions and the flexibility of semi-Markov chains for representing temporal structures. Efficient algorithms for computing characteristic distributions organized according to the intensity, interval and counting points of view are described. The proposed computational approach in conjunction with statistical inference algorithms previously proposed makes discrete hidden semi-Markov chains a powerful model for the analysis of samples of non-stationary discrete sequences. Copyright © 1999 John Wiley & Sons, Ltd.     

Initial and Final Backward and Forward Discrete Time Non-homogeneous Semi-Markov Credit Risk Models

In this paper we show how it is possible to construct an efficient Migration models in the study of credit risk problems presented in Jarrow et al. (Rev Financ Stud 10:481–523, 1997) with Markov environment. Recently it was introduced the semi-Markov process in the migration models (D’Amico et al. Decis Econ Finan 28:79–93, 2005a). The introduction of semi-Markov processes permits to overtake some of the Markov constraints given by the dependence of transition probabilities on the duration into a rating category. In this paper, it is shown how it is possible to take into account simultaneously backward and forward processes at beginning and at the end of the time in which the credit risk model is observed. With such a generalization, it is possible to consider what happens inside the time after the first transition and before the last transition where the problem is studied. This paper generalizes other papers presented before. The model is presented in a discrete time environment.     

Markovian modeling for dependent interrecurrence times in bladder cancer

A methodology to model a process in which repeated events occur is presented. The context is the evolution of non-muscle-invasive bladder carcinoma (NMIBC), characterized by recurrent relapses. It is based on the statistical flowgraph approach, a technique specifically suited for semi-Markov processes. A very useful feature of the flowgraph framework is that it naturally incorporates the management of censored data. However, this approach presents two difficulties with the process to be modeled. On one hand, the management of covariates is not straightforward. However, it is of great interest to know how the characteristics of a certain patient influence the evolution of the disease. On the other hand, repeated events on the same subject are generally not independent, in which case the semi-Markov framework is not sufficient because the semi-Markov assumption implies independence among waiting time distributions. We solve this issue by extending the flowgraph methodology using the Markovian arrival process (MAP), which does successfully model the dependence between events. Along the way, we provide a procedure to consider covariates and censored times in MAPs, a pending task needed in this field. In short, we have managed to extend the flowgraph methodology beyond the semi-Markovian framework, simplifying the incorporation of covariates and keeping the management of censored times. All of which has allowed us to build a multistate model of the evolution of NMIBC. The developed model allows us to compute the Survival function for any evolution of a patient with specific clinic-pathological characteristics in this primary tumor.     

A Non-Homogeneous Continuous Time Semi-Markov Model for the Study of Accumulated Claim Process

The accumulated claim process is the summed total of all claims starting from time t. The semi-Markov environment, at authors’ opinion, is able to follow the evolution of this process. In the paper a continuous time non-homogeneous semi-Markov model with a denumerable set of states will be used to follow the stochastic evolution of the accumulated claim process.     

半马氏过程的一维分布及构造

本文求出了半马氏过程跳跃链的转移概率,给出了半马氏过程的逗留时间分布和一维分布,构造了半马氏过程$X(t,\omega)$,最后证明了半马氏过程的两种定义是等价的.     

Bounds and approximations for continuous-time Markovian transition probabilities and large systems

We propose new bounds and approximations for the transition probabilities of a continuous-time Markov process with finite but large state-space. The bounding and approximating procedures have been exposed in another paper [S. Mercier, Numerical bounds for semi-Markovian quantities and applications to reliability, in revision for Methodology and Computing in Applied Probability] in the more general context of a continuous-time semi-Markov process with countable state-space. Such procedures are here specialized to the Markovian finite case, leading to much simpler algorithms. The aim of this paper is to test such algorithms versus other algorithms from the literature near from ours, such as forward Euler approximation, external uniformization and a finite volume method from [C. Cocozza-Thivent, R. Eymard, Approximation of the marginal distributions of a semi-Markov process using a finite volume scheme, ESAIM: M2AN 38(5) (2004) 853–875].     

Numerical Treatment of Homogeneous Semi-Markov Processes in Transient Case–a Straightforward Approach

This paper presents the numerical solution of the process evolution equation of a homogeneous semi-Markov process (HSMP) with a general quadrature method. Furthermore, results that justify this approach proving that the numerical solution tends to the evolution equation of the continuous time HSMP are given. The results obtained generalize classical results on integral equation numerical solutions applying them to particular kinds of integral equation systems. A method for obtaining the discrete time HSMP is shown by applying a very particular quadrature formula for the discretization. Following that, the problem of obtaining the continuous time HSMP from the discrete one is considered. In addition, the discrete time HSMP in matrix form is presented and the fact that the solution of the evolution equation of this process always exists is proved. Afterwards, an algorithm for solving the discrete time HSMP is given. Finally, a simple application of the HSMP is given for a real data social security example.     

On the construction of component importance measures for semi-Markov systems

In this paper we consider semi-Markov reliability models of systems with discrete state space in a setup general enough to cover systems with maintenance and repair. The systems are assumed to consist of several components which can either be up or down in each state. In this framework we propose two different types of component importance measures which are based on transition rates and interval availability, respectively. For these importance measures we study both the time-dependent and the steady state situation, and express them in terms of quantities easily calculated from the building blocks of the semi-Markov process.     

Assessing Markov property in multistate transition models with applications to credit risk modeling

Multistate transition models are increasingly used in credit risk applications as they allow us to quantify the evolution of the process among different states. If the process is Markov, analysis and prediction are substantially simpler, so analysts would like to use these models if they are applicable. In this paper, we develop a procedure for assessing the Markov hypothesis and discuss different ways of implementing the test procedure. One issue when sample size is large is that the statistical test procedures will detect even small deviations from the Markov model when these differences are not of practical interest. To address this problem, we propose an approach to formulate and test the null hypothesis of “weak non‐Markov.” The situation where the transition probabilities are heterogeneous is also examined, and approaches to accommodate this case are indicated. Simulation studies are used extensively to study the properties of the procedures, and two applications are to illustrate the results.     

Reliability of manufacturing equipment in complex environments

We present two stochastic failure models for the reliability evaluation of manufacturing equipment that degrades due to its complex operating environment. The first model examines the case when the environment is a temporally nonhomogeneous continuous-time Markov chain, and the second assumes the environment is a temporally homogeneous semi-Markov process on a finite space. Derived are transform expressions for the lifetime distributions. A few examples are provided to illustrate the main results.