Hidden semi-Markov model-based methodology for multi-sensor equipment health diagnosis and prognosis

This paper presents an integrated platform for multi-sensor equipment diagnosis and prognosis. This integrated framework is based on hidden semi-Markov model (HSMM). Unlike a state in a standard hidden Markov model (HMM), a state in an HSMM generates a segment of observations, as opposed to a single observation in the HMM. Therefore, HSMM structure has a temporal component compared to HMM. In this framework, states of HSMMs are used to represent the health status of a component. The duration of a health state is modeled by an explicit Gaussian probability function. The model parameters (i.e., initial state distribution, state transition probability matrix, observation probability matrix, and health-state duration probability distribution) are estimated through a modified forward–backward training algorithm. The re-estimation formulae for model parameters are derived. The trained HSMMs can be used to diagnose the health status of a component. Through parameter estimation of the health-state duration probability distribution and the proposed backward recursive equations, one can predict the useful remaining life of the component. To determine the “value” of each sensor information, discriminant function analysis is employed to adjust the weight or importance assigned to a sensor. Therefore, sensor fusion becomes possible in this HSMM based framework.     

Nonzero-sum semi-Markov games with the expected average payoffs

Nonzero-sum ergodic semi-Markov games with Borel state spaces are studied. An equilibrium theorem is proved in the class of correlated stationary strategies using public randomization. Under some additivity assumption concerning the transition probabilities stationary Nash equilibria are also shown to exist.Received: October 2004 / Revised: January 2005     

Coordinated production and inspection in a tandem system

We study the coordination of production and quality control in a tandem-queue system. There are two stages, with a single server at stage one that can engage in processing an item, or inspecting the produced item, or staying idle; whereas the second stage represents the aggregate of the rest of the production facility. We focus on the optimal control of the first stage, where both the production and inspection times follow general distributions. We formulate a semi-Markov decision program with a long-run average objective, and derive the stationary optimal policy to control and coordinate the production, inspection, and idling processes. We show that there exists a threshold valuei ^–, such that under the optimal policy, once the threshold is reached, production should be suspended at the first stage; and this leads naturally toi ^–+1 being the required buffer capacity between the two stages.Supported in part by NSF Grant MDI-9523029.Supported in part by HKUST Grant DAG95/96.BM52.     

A Markov Renewal Based Model for Wireless Networks

We propose a queueing network model which can be used for the integration of the mobility and teletraffic aspects that are characteristic of wireless networks. In the general case, the model is an open network of infinite server queues where customers arrive according to a non-homogeneous Poisson process. The movement of a customer in the network is described by a Markov renewal process. Moreover, customers have attributes, such as a teletraffic state, that are driven by continuous time Markov chains and, therefore, change as they move through the network. We investigate the transient and limit number of customers in disjoint sets of nodes and attributes. These turn out to be independent Poisson random variables. We also calculate the covariances of the number of customers in two sets of nodes and attributes at different time epochs. Moreover, we conclude that the arrival process per attribute to a node is the sum of independent Poisson cluster processes and derive its univariate probability generating function. In addition, the arrival process to an outside node of the network is a non-homogeneous Poisson process. We illustrate the applications of the queueing network model and the results derived in a particular wireless network.     

Optimal replacement policies for two-unit machines with increasing running costs 1

A machine consists of two stochastically failing units. Failure of either of the units causes a failure of the machine and the failed unit has to be replaced immediately. Associated with the units are running costs which increase with the age of the unit because of increasing maintenance costs, decreasing output, etc.A preventive replacement policy is proposed under which, at failure points, we also replace the second unit if its age exceeds a predetermined control limit. It is proved that, for two identical units with exponential life-time distributions and linear running costs, this policy is optimal and the optimal control limit is calculated. In an additional model we take into consideration the length of time it takes to replace one unit or both units.The method of solution is a variation of dynamic semi-Markov programming. Analytical results are obtained and the influence of the various parameters on them is investigated. Finally, we study the saving due to our policy in comparison with a policy in which only failed units are replaced.     

Sequential tests on a multinomial distribution and a Markov chain

In an earlier paper, a sequential test of the equality of probabilities in a multinomial distribution was proposed, and compared to fixed sample size tests in the special case of two categories. In the present paper, the test is generalized to testing any simple hypothesis about a multinomial distribution, and compared to fixed sample size tests when the number of categories is large. The method of construction of the test is used to construct tests with precisely the desired level of significance, for multinomial and other problems, including Markov chains.     

Stability of stochastic jump-parameter semi-Markov linear systems of differential equations

The asymptotic stability of stochastic Itô-type jump-parameter semi-Markov systems of linear differential equations is examined. A system of integral matrix equations is derived which has the property that the existence of a positive definite solution of the system implies the asymptotic stability of the stochastic semi-Markov system. Finally, an illustrative example is presented.     

Estimation in Stationary Markov Renewal Processes,with Application to Earthquake Forecasting in Turkey

Consider a process in which different events occur, with random inter-occurrence times. In Markov renewal processes as well as in semi-Markov processes, the sequence of events is a Markov chain and the waiting distributions depend only on the types of the last and the next event. Suppose that the state-space is finite and that the process started far in the past, achieving stationary. Weibull distributions are proposed for the waiting times and their parameters are estimated jointly with the transition probabilities through maximum likelihood, when one or several realizations of the process are observed over finite windows. The model is illustrated with data of earthquakes of three types of severity that occurred in Turkey during the 20th century.AMS 2000 Subject Classification: 60K20     

Numerical Solution of non-Homogeneous Semi-Markov Processes in Transient Case*

In this article a numerical solution for the evolution equation of a continuous time non-homogeneous semi-Markov process (NHSMP) is obtained using a quadrature method. The paper, after a short introduction to continuous time NHSMP, presents the numerical solution of the process evolution equation with a general quadrature method. Furthermore, the paper gives results that justify this approach, proving that the numerical solution tends to the evolution equation of the continuous time NHSMP. Moreover, the formulae related to some specific quadrature methods are given and a method for obtaining the discrete time NHSMP by applying a very particular quadrature formula for the discretization is shown. In this way the relation between the continuous and discrete time NHSMP is proved. Then, the problem of obtaining the continuous time NHSMP from the discrete one is considered. This problem is solved showing that the discrete process converges in law to the continuous one if the discretized time interval tends to zero. In addition, the discrete time NHSMP in matrix form is presented, and the fact that the solution to this process always exists is proved. Finally, an algorithm for solving the discrete time NHSMP is given. To illustrate the use of this algorithm for a discrete NHSMP, an example in the area of finance is presented.     

��ɢ�����Ϸ���ģ���е�����������

??This paper considers the expected penalty functions for a discrete semi-Markov risk model, which includes several existing risk models such as the compound binomial model (with time-correlated claims) and the compound Markov binomial model (with time-correlated claims) as special cases. Recursive formulae and the initial values for the discounted free penalty functions are derived in the two-state model by an easy method. We also give some applications of our results.