The steady‐state probabilities for regenerative semi‐Markov processes with application to prevention and screening

There is a growing interest in planning and implementing broad‐scale clinical trials with a focus on prevention and screening. Often, the data‐generating mechanism for such experiments can be viewed as a semi‐Markov process. In this communication, we develop general expressions for the steady‐state probabilities for regenerative semi‐Markov processes. Hence, the probability of being in a certain state at the time of recruitment to a clinical trial can be calculated. An application to breast cancer prevention is demonstrated. Copyright © 1999 John Wiley & Sons, Ltd.     

Profile‐based push models in manpower planning

A methodology is presented to deal with heterogeneity due to observable variables in modeling personnel systems. For a manpower system, a model based on personnel profiles is introduced. The proposed algorithm concerns an analysis of the evolution of personnel profiles under time‐discrete Markov assumptions. In this way, based on an historical personnel database, the estimation of transition probabilities of profiles as well as the computation of forecasts on the evolution of the manpower system can be found. Based on the presented procedure, a policy supporting software package has been developed for the Belgian Federal Government (research project financed by Federaal Wetenschapsbeleid). Copyright © 2007 John Wiley & Sons, Ltd.     

Optimal stopping time on discounted semi-Markov processes

This paper attempts to study the optimal stopping time for semi- Markov processes (SMPs) under the discount optimization criteria with unbounded cost rates. In our work, we introduce an explicit construction of the equivalent semi-Markov decision processes (SMDPs). The equivalence is embodied in the expected discounted cost functions of SMPs and SMDPs, that is, every stopping time of SMPs can induce a policy of SMDPs such that the value functions are equal, and vice versa. The existence of the optimal stopping time of SMPs is proved by this equivalence relation. Next, we give the optimality equation of the value function and develop an effective iterative algorithm for computing it. Moreover, we show that the optimal and ε-optimal stopping time can be characterized by the hitting time of the special sets. Finally, to illustrate the validity of our results, an example of a maintenance system is presented in the end.     

经2状态马氏修正的Poisson过程的极大似然估计

近年来。隐马氏模型成为研究相依随机变量的一个十分有用的工具。应用过程中的一个很重要的问题是如何对隐马氏模型的参数进行估计。一般使用的方法是将连续时间隐马氏模型的问题转化为离散时间隐马氏模型的问题来讨论。本文用此方法讨论一类连续时间隐马氏模型——状态个数为2的经马氏链修正的Poisson过程的极大似然估计及其算法。此类模型被广泛用来对复杂通信网络的通信流进行建模。     

Future pricing through homogeneous semi‐Markov processes

An Erratum for this article has been published in Applied Stochastic Models in Business and Industry 2005; (in press) This paper presents a future pricing model based on the discrete time homogeneous semi‐Markov process (DTHSMP). The model is adapted to the real data of the Italian primary future stock index. After showing the pricing model, the DTHSMP solution is given. The solution of the semi‐Markov process gives, for each period of the considered horizon time, and for each starting state, the probability distribution of the future price. Copyright © 2005 John Wiley & Sons, Ltd.     

The augmented semi‐Markov system and its asymptotic behaviour

This paper suggests a generalized semi‐Markov model for manpower planning, which could be adopted in cases of unavailability of candidates with the desired qualifications/experience, as well as in cases where an organization provides training opportunities to its personnel. In this context, we incorporate training classes into the framework of a non‐homogeneous semi‐Markov system and we introduce an additional, external semi‐Markov system providing the former with potential recruits. For the model above, referred to as the Augmented Semi‐Markov System, we derive the equations that reflect the expected number of persons in each grade and we also investigate its limiting population structure. An illustrative example is provided. Copyright © 2010 John Wiley & Sons, Ltd.     

Regenerative simulation of TES processes

Regenerative simulation has become a familiar and established tool for simulation-based estimation. However, many applications (e.g., traffic in high-speed communications networks) call for autocorrelated stochastic models to which traditional regenerative theory is not directly applicable. Consequently, extensions of regenerative simulation to dependent time series is increasingly gaining in theoretical and practical interest, with Markov chains constituting an important case. Fortunately, a regenerative structure can be identified in Harris-recurrent Markov chains with minor modification, and this structure can be exploited for standard regenerative estimation. In this paper we focus on a versatile class of Harris-recurrent Markov chains, called TES (Transform-Expand-Sample). TES processes can generate a variety of sample paths with arbitrary marginal distributions, and autocorrelation functions with a variety of functional forms (monotone, oscillating and alternating). A practical advantage of TES processes is that they can simultaneously capture the first and second order statistics of empirical sample paths (raw field measurements). Specifically, the TES modeling methodology can simultaneously match the empirical marginal distribution (histogram), as well as approximate the empirical autocorrelation function. We explicitly identify regenerative structures in TES processes and proceed to address efficiency and accuracy issues of prospective simulations. To show the efficacy of our approach, we report on a TES/M/1 case study. In this study, we used the likelihood ratio method to calculate the mean waiting time performance as a function of the regenerative structure and the intrinsic TES parameter controlling burstiness (degree of autocorrelation) in the arrival process. The score function method was used to estimate the corresponding sensitivity (gradient) with respect to the service rate. Finally, we demonstrated the importance of the particular regenerative structure selected in regard to the estimation efficiency and accuracy induced by the regeneration cycle length.     

Parameter Estimation in Hidden Markov Models With Intractable Likelihoods Using Sequential Monte Carlo

We propose sequential Monte Carlo-based algorithms for maximum likelihood estimation of the static parameters in hidden Markov models with an intractable likelihood using ideas from approximate Bayesian computation. The static parameter estimation algorithms are gradient-based and cover both offline and online estimation. We demonstrate their performance by estimating the parameters of three intractable models, namely the α-stable distribution, g-and-k distribution, and the stochastic volatility model with α-stable returns, using both real and synthetic data.     

A standby system with two types of repair persons

A continuously monitored one‐unit system, backed by an identical standby unit, is perfectly repaired by an in‐house repair person, if achievable within a random or deterministic patience time (DPT), or else by a visiting expert, who repairs one or all failed units before leaving. We study four models in terms of the limiting availability and limiting profit per unit time, using semi‐Markov processes, when all distributions are exponential. We show that a DPT is preferable to a random patience time, and we characterize conditions under which the expert should repair multiple failed units (rather than only one failed unit) during each visit. We also extend the method when life‐ and repair times are non‐exponential. Copyright © 2009 John Wiley & Sons, Ltd.     

Stationary availability of a semi‐Markov system with random maintenance

We consider a reparable system with a finite state space, evolving in time according to a semi‐Markov process. The system is stopped for it to be preventively maintained at random times for a random duration. Our aim is to find the preventive maintenance policy that optimizes the stationary availability, whenever it exists. The computation of the stationary availability is based on the fact that the above maintained system evolves according to a semi‐regenerative process. As for the optimization, we observe on numerical examples that it is possible to limit the study to the maintenance actions that begin at deterministic times. We demonstrate this result in a particular case and we study the deterministic maintenance policies in that case. In particular, we show that, if the initial system has an increasing failure rate, the maintenance actions improve the stationary availability if and only if they are not too long on the average, compared to the repairs ( a bound for the mean duration of the maintenance actions is provided). On the contrary, if the initial system has a decreasing failure rate, the maintenance policy lowers the stationary availability. A few other cases are studied. Copyright © 2000 John Wiley & Sons, Ltd.     

Learning genetic algorithm parameters using hidden Markov models

Genetic algorithms (GAs) are routinely used to search problem spaces of interest. A lesser known but growing group of applications of GAs is the modeling of so-called “evolutionary processes”, for example, organizational learning and group decision-making. Given such an application, we show it is possible to compute the likely GA parameter settings given observed populations of such an evolutionary process. We examine the parameter estimation process using estimation procedures for learning hidden Markov models, with mathematical models that exactly capture expected GA behavior. We then explore the sampling distributions relevant to this estimation problem using an experimental approach.     

Spatiotemporal Models for Gaussian Areal Data

We introduce a class of spatiotemporal models for Gaussian areal data. These models assume a latent random field process that evolves through time with random field convolutions; the convolving fields follow proper Gaussian Markov random field (PGMRF) processes. At each time, the latent random field process is linearly related to observations through an observational equation with errors that also follow a PGMRF. The use of PGMRF errors brings modeling and computational advantages. With respect to modeling, it allows more flexible model structures such as different but interacting temporal trends for each region, as well as distinct temporal gradients for each region. Computationally, building upon the fact that PGMRF errors have proper density functions, we have developed an efficient Bayesian estimation procedure based on Markov chain Monte Carlo with an embedded forward information filter backward sampler (FIFBS) algorithm. We show that, when compared with the traditional one-at-a-time Gibbs sampler, our novel FIFBS-based algorithm explores the posterior distribution much more efficiently. Finally, we have developed a simulation-based conditional Bayes factor suitable for the comparison of nonnested spatiotemporal models. An analysis of the number of homicides in Rio de Janeiro State illustrates the power of the proposed spatiotemporal framework.

Supplemental materials for this article are available online in the journal’s webpage.     

Branching Markov processes and related asymptotics

Models for Markov processes indexed by a branching process are presented. The new class of models is referred to as the branching Markov process (BMP). The law of large numbers and a central limit theorem for the BMP are established. Bifurcating autoregressive processes (BAR) are special cases of the general BMP model discussed in the paper. Applications to parameter estimation are also presented.     

Toward a formalism of modeling and simulation using model theory

This article proposes a Modeling and Simulation (M&S) formalism using Model Theory. The article departs from the premise that M&S is the science that studies the nature of truth using models and simulations. Truth in models and simulations is relative as they seek to answer specific modeling questions. Consequently, truth in M&S is relative because every model is a purposeful abstraction of reality. We use Model Theory to express the proposed formalism because it is built from the premise that truth is relative. The proposed formalism allows us to: (1) deduce formal definitions and explanations of areas of study in M&S, including conceptual modeling, validity, and interoperability, and (2) gain insight into which tools can be used to semi‐automate validation and interoperation processes. © 2013 Wiley Periodicals, Inc. Complexity 19: 56–63, 2014     

Conditionally heteroscedastic factorial HMMs for time series in finance

In this article, we develop a new approach within the framework of asset pricing models that incorporates two key features of the latent volatility: co‐movement among conditionally heteroscedastic financial returns and switching between different unobservable regimes. By combining latent factor models with hidden Markov chain models we derive a dynamical local model for segmentation and prediction of multivariate conditionally heteroscedastic financial time series. We concentrate more precisely on situations where the factor variances are modelled by univariate generalized quadratic autoregressive conditionally heteroscedastic processes. The expectation maximization algorithm that we have developed for the maximum likelihood estimation is based on a quasi‐optimal switching Kalman filter approach combined with a generalized pseudo‐Bayesian approximation, which yield inferences about the unobservable path of the common factors, their variances and the latent variable of the state process. Extensive Monte Carlo simulations and preliminary experiments obtained with daily foreign exchange rate returns of eight currencies show promising results. Copyright © 2007 John Wiley & Sons, Ltd.     

Bayesian estimation of stochastic volatility models based on OU processes with marginal Gamma law

This paper discusses practical Bayesian estimation of stochastic volatility models based on OU processes with marginal Gamma laws. Estimation is based on a parameterization which is derived from the Rosiński representation, and has the advantage of being a non-centered parameterization. The parameterization is based on a marked point process, living on the positive real line, with uniformly distributed marks. We define a Markov chain Monte Carlo (MCMC) scheme which enables multiple updates of the latent point process, and generalizes single updating algorithm used earlier. At each MCMC draw more than one point is added or deleted from the latent point process. This is particularly useful for high intensity processes. Furthermore, the article deals with superposition models, where it discuss how the identifiability problem inherent in the superposition model may be avoided by the use of a Markov prior. Finally, applications to simulated data as well as exchange rate data are discussed.     

Extended Gauss–Markov Theorem for Nonparametric Mixed-Effects Models

The Gauss–Markov theorem provides a golden standard for constructing the best linear unbiased estimation for linear models. The main purpose of this article is to extend the Gauss–Markov theorem to include nonparametric mixed-effects models. The extended Gauss–Markov estimation (or prediction) is shown to be equivalent to a regularization method and its minimaxity is addressed. The resulting Gauss–Markov estimation serves as an oracle to guide the exploration for effective nonlinear estimators adaptively. Various examples are discussed. Particularly, the wavelet nonparametric regression example and its connection with a Sobolev regularization is presented.     

The Alive Particle Filter and Its Use in Particle Markov Chain Monte Carlo

In the following article, we investigate a particle filter for approximating Feynman–Kac models with indicator potentials and we use this algorithm within Markov chain Monte Carlo (MCMC) to learn static parameters of the model. Examples of such models include approximate Bayesian computation (ABC) posteriors associated with hidden Markov models (HMMs) or rare-event problems. Such models require the use of advanced particle filter or MCMC algorithms to perform estimation. One of the drawbacks of existing particle filters is that they may “collapse,” in that the algorithm may terminate early, due to the indicator potentials. In this article, using a newly developed special case of the locally adaptive particle filter, we use an algorithm that can deal with this latter problem, while introducing a random cost per-time step. In particular, we show how this algorithm can be used within MCMC, using particle MCMC. It is established that, when not taking into account computational time, when the new MCMC algorithm is applied to a simplified model it has a lower asymptotic variance in comparison to a standard particle MCMC algorithm. Numerical examples are presented for ABC approximations of HMMs.     

Maximum likelihood estimation for stochastic volatility in mean models with heavy‐tailed distributions

In this article, we introduce a likelihood‐based estimation method for the stochastic volatility in mean (SVM) model with scale mixtures of normal (SMN) distributions. Our estimation method is based on the fact that the powerful hidden Markov model (HMM) machinery can be applied in order to evaluate an arbitrarily accurate approximation of the likelihood of an SVM model with SMN distributions. Likelihood‐based estimation of the parameters of stochastic volatility models, in general, and SVM models with SMN distributions, in particular, is usually regarded as challenging as the likelihood is a high‐dimensional multiple integral. However, the HMM approximation, which is very easy to implement, makes numerical maximum of the likelihood feasible and leads to simple formulae for forecast distributions, for computing appropriately defined residuals, and for decoding, that is, estimating the volatility of the process. Copyright © 2017 John Wiley & Sons, Ltd.     

������ɷ��Ԫ��̬�ֲ������ļ�����Ȼ����

??Hidden Markov model is widely used in statistical modeling of time, space and state transition data. The definition of hidden Markov multivariate normal distribution is given. The principle of using cluster analysis to determine the hidden state of observed variables is introduced. The maximum likelihood estimator of the unknown parameters in the model is derived. The simulated observation data set is used to test the estimation effect and stability of the method. The characteristic is simple classical statistical inference such as cluster analysis and maximum likelihood estimation. The method solves the parameter estimation problem of complex statistical models.