A computational procedure for the asymptotic analysis of homogeneous semi-Markov processes

A new algorithm for classifying the states of a homogeneous Markov chain having finitely many states is presented, which enables the investigation of the asymptotic behavior of semi-Markov processes in which the Markov chain is embedded. An application of the algorithm to a social security problem is also presented.     

On mean reward variance in semi-Markov processes

As an extension of the discrete-time case, this note investigates the variance of the total cumulative reward for the embedded Markov chain of semi-Markov processes. Under the assumption that the chain is aperiodic and contains a single class of recurrent states recursive formulae for the variance are obtained which show that the variance growth rate is asymptotically linear in time. Expressions are provided to compute this growth rate.     

Computational methods for discrete hidden semi-Markov chains

We propose a computational approach for implementing discrete hidden semi-Markov chains. A discrete hidden semi-Markov chain is composed of a non-observable or hidden process which is a finite semi-Markov chain and a discrete observable process. Hidden semi-Markov chains possess both the flexibility of hidden Markov chains for approximating complex probability distributions and the flexibility of semi-Markov chains for representing temporal structures. Efficient algorithms for computing characteristic distributions organized according to the intensity, interval and counting points of view are described. The proposed computational approach in conjunction with statistical inference algorithms previously proposed makes discrete hidden semi-Markov chains a powerful model for the analysis of samples of non-stationary discrete sequences. Copyright © 1999 John Wiley & Sons, Ltd.     

M/G/1排队系统的性能灵敏度分析

非Markov型排除系统经常被用来作为某些实际工程问题（如通讯网络）的研究模型,对于一般的M／G／1排队系统,本文通过研究其嵌入Markov链,讨论了系统的稳态性能灵敏度分析问题,并给出用嵌入Markov链的势能表示的稳态性能灵敏度公式,由于嵌入Markov链要比描述其系统状态的半Markov过程简单得多,故本文的结果对M／G／1排队系统的性能灵敏度仿真计算及系统的优化,都将带来极大的方便。     

Estimating Hidden Semi-Markov Chains From Discrete Sequences

This article addresses the estimation of hidden semi-Markov chains from nonstationary discrete sequences. Hidden semi-Markov chains are particularly useful to model the succession of homogeneous zones or segments along sequences. A discrete hidden semi-Markov chain is composed of a nonobservable state process, which is a semi-Markov chain, and a discrete output process. Hidden semi-Markov chains generalize hidden Markov chains and enable the modeling of various durational structures. From an algorithmic point of view, a new forward-backward algorithm is proposed whose complexity is similar to that of the Viterbi algorithm in terms of sequence length (quadratic in the worst case in time and linear in space). This opens the way to the maximum likelihood estimation of hidden semi-Markov chains from long sequences. This statistical modeling approach is illustrated by the analysis of branching and flowering patterns in plants.     

半马氏过程的一维分布及构造

本文求出了半马氏过程跳跃链的转移概率,给出了半马氏过程的逗留时间分布和一维分布,构造了半马氏过程$X(t,\omega)$,最后证明了半马氏过程的两种定义是等价的.     

The two-element system with one restoring device optimum maintenance

The paper considers the problem of the optimum preventive maintenance of the system with two elements and one re¬storing device. The system 's behaviour is described by a semi-Markov process with complex descrete and continuous set of states. Whether the preventive maintenance is performed depends on the state of the system's elements and the level of past life of the element which performs the duties of the main element. The paper contains the solution to the system of integral equations relative to the stationary distribution of a Markov chain included in given semi-Markov process. This results makes it possible to find various-stationary functionals on the process trajectories and lead the task of optimum control to searching the extreme value of given function of two real variables     

基于半马尔科夫过程的逻辑门可靠性分析

动态故障树分析方法是在静态故障树的基础上拓展而来的自上而下的图形化演绎技术,可以很好地对具有复杂失效行为和交互作用的系统进行建模,进而分析系统的可靠性。本文从动态故障树逻辑门的可靠性建模与分析入手,结合半马尔科夫过程原理,将动态逻辑门转化为半马尔科夫链。其次给出在半马尔科夫链中动态逻辑门输出事件的发生概率和系统可靠性的计算公式。提出各种逻辑门到半马尔科夫链的通用转化模型,通过更改通用模型中的相关参数,将逻辑门转化为半马尔科夫链。最后,基于半马尔科夫过程求解动态逻辑门输出事件的发生概率,以动态优先与门、顺序相关门和备件门为例,并给出系统可靠性的计算公式。     

A characterization for mixtures of semi-Markov processes

Mixtures of recurrent semi-Markov processes are characterized through a partial exchangeability condition of the array of successor states and holding times. A stronger invariance condition on the joint law of successor states and holding times leads to mixtures of Markov laws.     

非负费用折扣半马氏决策过程

本文考虑可数状态非负费用的折扣半马氏决策过程.首先在给定半马氏决策核和策略下构造一个连续时间半马氏决策过程,然后用最小非负解方法证明值函数满足最优方程和存在ε-最优平稳策略,并进一步给出最优策略的存在性条件及其一些性质.最后,给出了值迭代算法和一个数值算例.     

The crossing barrier of a non-homogeneous semi-Markov chain

In this paper, we derive the probability distribution function of the first entry time in a time varying subset of the phase space of a non-homogeneous semi-Markov chain. Deterministic and stochastic time varying subsets are considered and results with a ready algorithmic implementation are provided. A possible application is debated by defining and analysing deterministic and stochastic time variable level crossing order for semi-Markov chains.     

Semi-Markov dynamic programming approach to competitive bidding with state space reduction considerations

An infinite horizon semi-Markov dynamic programming model is formulated to compute the optimal bidding strategy for a firm using multiple limited resources with idle costs, sequentially bidding in each period for one of the multiple types of projects with different durations and resource requirements.An oligopolistic market with generally stable environmental conditions, in which the lowest bidder secures the project, is considered. Previous bid results are assumed to be known prior to submitting a subsequent bid.     

First passage models for denumerable semi-Markov decision processes with nonnegative discounted costs

This paper considers a first passage model for discounted semi-Markov decision processes with denumerable states and nonnegative costs.The criterion to be optimized is the expected discounted cost incurred during a first passage time to a given target set.We first construct a semi-Markov decision process under a given semi-Markov decision kernel and a policy.Then,we prove that the value function satisfies the optimality equation and there exists an optimal(or e-optimal) stationary policy under suitable conditions by using a minimum nonnegative solution approach.Further we give some properties of optimal policies.In addition,a value iteration algorithm for computing the value function and optimal policies is developed and an example is given.Finally,it is showed that our model is an extension of the first passage models for both discrete-time and continuous-time Markov decision processes.     

基于MRS-GARCH模型的中国股市波动率估计与预测

基于误差项服从正态分布、t分布、广义误差分布的GARCH族模型和MRS-GARCH模型对中国股市波动的结构变化特征进行了实证研究。结果表明,中国股市存在显著的高、低波动状态,两种波动状态的ARCH和GARCH项系数存在较大差异;高、低波动状态均具有较长的持续时间,低波动状态的持续时间长于高波动状态的持续时间,且中国股市更易于从高波动状态转向低波动状态;MRS-GARCH模型预测效果总体上优于GARCH族模型,基于正态分布的MRS-GARCH模型短期预测效果较好。     

Stochastic Processes with Age-Dependent Transition Rates

We study stochastic processes with age-dependent transition rates. A typical example of such a process is a semi-Markov process which is completely determined by the holding time distributions in each state and the transition probabilities of the embedded Markov chain. The process we construct generalizes semi-Markov processes. One important feature of this process is that unlike semi-Markov processes the transition probabilities of this process are age-dependent. Under certain condition we establish the Feller property of the process. Finally, we compute the limiting distribution of the process.     

Sufficient Stochastic Maximum Principle for the Optimal Control of Semi-Markov Modulated Jump-Diffusion with Application to Financial Optimization

The finite state semi-Markov process is a generalization over the Markov chain in which the sojourn time distribution is any general distribution. In this article, we provide a sufficient stochastic maximum principle for the optimal control of a semi-Markov modulated jump-diffusion process in which the drift, diffusion, and the jump kernel of the jump-diffusion process is modulated by a semi-Markov process. We also connect the sufficient stochastic maximum principle with the dynamic programming equation. We apply our results to finite horizon risk-sensitive control portfolio optimization problem and to a quadratic loss minimization problem.     

Maximum likelihood estimation for general hidden semi-Markov processes with backward recurrence time dependence

This paper concerns the study of asymptotic properties of the maximum likelihood estimator (MLE) for the general hidden semi-Markov model (HSMM) with backward recurrence time dependence. By transforming the general HSMM into a general hidden Markov model, we prove that under some regularity conditions, the MLE is strongly consistent and asymptotically normal. We also provide useful expressions for asymptotic covariance matrices, involving the MLE of the conditional sojourn times and the embedded Markov chain of the hidden semi-Markov chain. Bibliography: 17 titles.     

Existence theorems for a semi-Markov process with an arbitrary set of states

We introduce a semi-Markov process x_t with an arbitrary measurable set of states; we formulate conditions for the absence of second order discontinuities in the trajectories of the process sufficient for realization of the semi-Markov process x_t as a probability measure in the space of step-functions. We conclude with a study of the resulting model.     

Analysis of a single server queue with semi-Markovian service interruption

In this paper we analyze a discrete-time single server queue where the service time equals one slot. The numbers of arrivals in each slot are assumed to be independent and identically distributed random variables. The service process is interrupted by a semi-Markov process, namely in certain states the server is available for service while the server is not available in other states. We analyze both the transient and steady-state models. We study the generating function of the joint probability of queue length, the state and the residual sojourn time of the semi-Markov process. We derive a system of Hilbert boundary value problems for the generating functions. The system of Hilbert boundary value problems is converted to a system of Fredholm integral equations. We show that the system of Fredholm integral equations has a unique solution. This revised version was published online in June 2006 with corrections to the Cover Date.