经2状态马氏修正的Poisson过程的极大似然估计

近年来。隐马氏模型成为研究相依随机变量的一个十分有用的工具。应用过程中的一个很重要的问题是如何对隐马氏模型的参数进行估计。一般使用的方法是将连续时间隐马氏模型的问题转化为离散时间隐马氏模型的问题来讨论。本文用此方法讨论一类连续时间隐马氏模型——状态个数为2的经马氏链修正的Poisson过程的极大似然估计及其算法。此类模型被广泛用来对复杂通信网络的通信流进行建模。     

The Alive Particle Filter and Its Use in Particle Markov Chain Monte Carlo

In the following article, we investigate a particle filter for approximating Feynman–Kac models with indicator potentials and we use this algorithm within Markov chain Monte Carlo (MCMC) to learn static parameters of the model. Examples of such models include approximate Bayesian computation (ABC) posteriors associated with hidden Markov models (HMMs) or rare-event problems. Such models require the use of advanced particle filter or MCMC algorithms to perform estimation. One of the drawbacks of existing particle filters is that they may “collapse,” in that the algorithm may terminate early, due to the indicator potentials. In this article, using a newly developed special case of the locally adaptive particle filter, we use an algorithm that can deal with this latter problem, while introducing a random cost per-time step. In particular, we show how this algorithm can be used within MCMC, using particle MCMC. It is established that, when not taking into account computational time, when the new MCMC algorithm is applied to a simplified model it has a lower asymptotic variance in comparison to a standard particle MCMC algorithm. Numerical examples are presented for ABC approximations of HMMs.     

Efficient sensitivity analysis in hidden markov models

Sensitivity analysis in hidden Markov models (HMMs) is usually performed by means of a perturbation analysis where a small change is applied to the model parameters, upon which the output of interest is re-computed. Recently it was shown that a simple mathematical function describes the relation between HMM parameters and an output probability of interest; this result was established by representing the HMM as a (dynamic) Bayesian network. To determine this sensitivity function, it was suggested to employ existing Bayesian network algorithms. Up till now, however, no special purpose algorithms for establishing sensitivity functions for HMMs existed. In this paper we discuss the drawbacks of computing HMM sensitivity functions, building only upon existing algorithms. We then present a new and efficient algorithm, which is specially tailored for determining sensitivity functions in HMMs.     

Generalisation of the Viterbi algorithm

The Viterbi algorithm, derived using dynamic programming techniques,is a maximum a posteriori (MAP) decoding method which was developedin the electrical engineering literature to be used in the analysisof hidden Markov models (HMMs). Given a particular HMM, theoriginal algorithm recovers the MAP state sequence underlyingany observation sequence generated from that model. This paperintroduces a generalization of the algorithm to recover, forarbitrary L, the top L most probable state sequences, with specialreference to its use in the area of automatic speech recognition.     

An S-Plus Implementation of Hidden Markov Models in Continuous Time

Abstract

Hidden Markov models (HMM) can be applied to the study of time varying unobserved categorical variables for which only indirect measurements are available. An S-Plus module to fit HMMs in continuous time to this type of longitudinal data is presented. Covariates affecting the transition intensities of the hidden Markov process or the conditional distribution of the measured response (given the hidden states of the process) are handled under a generalized regression framework. Users can provide C subroutines specifying the parameterization of the model to adapt the software to a wide variety of data types. HMM analysis using the S-Plus module is illustrated on a dataset from a prospective study of human papillomavirus infection in young women and on simulated data.     

Computational issues in parameter estimation for stationary hidden Markov models

The parameters of a hidden Markov model (HMM) can be estimated by numerical maximization of the log-likelihood function or, more popularly, using the expectation–maximization (EM) algorithm. In its standard implementation the latter is unsuitable for fitting stationary hidden Markov models (HMMs). We show how it can be modified to achieve this. We propose a hybrid algorithm that is designed to combine the advantageous features of the two algorithms and compare the performance of the three algorithms using simulated data from a designed experiment, and a real data set. The properties investigated are speed of convergence, stability, dependence on initial values, different parameterizations. We also describe the results of an experiment to assess the true coverage probability of bootstrap-based confidence intervals for the parameters.     

Online EM Algorithm for Hidden Markov Models

Online (also called “recursive” or “adaptive”) estimation of fixed model parameters in hidden Markov models is a topic of much interest in times series modeling. In this work, we propose an online parameter estimation algorithm that combines two key ideas. The first one, which is deeply rooted in the Expectation-Maximization (EM) methodology, consists in reparameterizing the problem using complete-data sufficient statistics. The second ingredient consists in exploiting a purely recursive form of smoothing in HMMs based on an auxiliary recursion. Although the proposed online EM algorithm resembles a classical stochastic approximation (or Robbins–Monro) algorithm, it is sufficiently different to resist conventional analysis of convergence. We thus provide limited results which identify the potential limiting points of the recursion as well as the large-sample behavior of the quantities involved in the algorithm. The performance of the proposed algorithm is numerically evaluated through simulations in the case of a noisily observed Markov chain. In this case, the algorithm reaches estimation results that are comparable to those of the maximum likelihood estimator for large sample sizes. The supplemental material for this article available online includes an appendix with the proofs of Theorem 1 and Corollary 1 stated in Section 4 as well as the MATLAB/OCTAVE code used to implement the algorithm in the case of a noisily observed Markov chain considered in Section 5.     

Cross-entropic learning of a machine for the decision in a partially observable universe

In this paper, we are interested in optimal decisions in a partially observable universe. Our approach is to directly approximate an optimal strategic tree depending on the observation. This approximation is made by means of a parameterized probabilistic law. A particular family of Hidden Markov Models (HMM), with input and output, is considered as a model of policy. A method for optimizing the parameters of these HMMs is proposed and applied. This optimization is based on the cross-entropic (CE) principle for rare events simulation developed by Rubinstein.     

Hidden Markov Models Training by a Particle Swarm Optimization Algorithm

In this work we consider the problem of Hidden Markov Models (HMM) training. This problem can be considered as a global optimization problem and we focus our study on the Particle Swarm Optimization (PSO) algorithm. To take advantage of the search strategy adopted by PSO, we need to modify the HMM's search space. Moreover, we introduce a local search technique from the field of HMMs and that is known as the Baum–Welch algorithm. A parameter study is then presented to evaluate the importance of several parameters of PSO on artificial data and natural data extracted from images.     

Parallel sequential Monte Carlo samplers and estimation of the number of states in a Hidden Markov Model

The majority of modelling and inference regarding Hidden Markov Models (HMMs) assumes that the number of underlying states is known a priori. However, this is often not the case and thus determining the appropriate number of underlying states for a HMM is of considerable interest. This paper proposes the use of a parallel sequential Monte Carlo samplers framework to approximate the posterior distribution of the number of states. This requires no additional computational effort if approximating parameter posteriors conditioned on the number of states is also necessary. The proposed strategy is evaluated on a comprehensive set of simulated data and shown to outperform the state of the art in this area: although the approach is simple, it provides good performance by fully exploiting the particular structure of the problem. An application to business cycle analysis is also presented.     

A higher order Markov model for analyzing covariate dependence

During the recent past, there has been a renewed interest in Markov chain for its attractive properties for analyzing real life data emerging from time series or longitudinal data in various fields. The models were proposed for fitting first or higher order Markov chains. However, there is a serious lack of realistic methods for linking covariate dependence with transition probabilities in order to analyze the factors associated with such transitions especially for higher order Markov chains. L.R. Muenz and L.V. Rubinstein [Markov models for covariate dependence of binary sequences, Biometrics 41 (1985) 91–101] employed logistic regression models to analyze the transition probabilities for a first order Markov model. The methodology is still far from generalization in terms of formulating a model for higher order Markov chains. In this study, it is aimed to provide a comprehensive covariate-dependent Markov model for higher order. The proposed model generalizes the estimation procedure for Markov models for any order. The proposed models and inference procedures are simple and the covariate dependence of the transition probabilities of any order can be examined without making the underlying model complex. An example from rainfall data is illustrated in this paper that shows the utility of the proposed model for analyzing complex real life problems. The application of the proposed method indicates that the higher order covariate dependent Markov models can be conveniently employed in a very useful manner and the results can provide in-depth insights to both the researchers and policymakers to resolve complex problems of underlying factors attributing to different types of transitions, reverse transitions and repeated transitions. The estimation and test procedures can be employed for any order of Markov model without making the theory and interpretation difficult for the common users.     

Hierarchic Markov processes and their applications in replacement models

In this paper a new notion of a hierarchic Markov process is introduced. It is a series of Markov decision processes called subprocesses built together in one Markov decision process called the main process. The hierarchic structure is specially designed to fit replacement models which in the traditional formulation as ordinary Markov decision processes are usually very large. The basic theory of hierarchic Markov processes is described and examples are given of applications in replacement models. The theory can be extended to fit a situation where the replacement decision depends on the quality of the new asset available for replacement.     

可列非齐次隐Markov模型的强大数定律

隐马尔科夫模型被广泛的应用于弱相依随机变量的建模,是研究神经生理学、发音过程和生物遗传等问题的有力工具。研究了可列非齐次隐 Markov 模型的若干性质,得到了这类模型的强大数定律,推广了有限非齐次马氏链的一类强大数定律。     

Rainflow analysis in Coastal Engineering using switching second order Markov models

The paper deals with the use of Markov and switching Markov chain models of turning points to reproduce random sets of sea states. The advantages of these models are emphasized and compared with existing models based on wave height records, indicating that long and short range and period cycles are included, while the wave height records ignore this important information from the point of view of damage accumulation. Existing models for first order Markov processes are extended to the case of second order processes and closed formulas are given to derive the rainflow matrices of these processes. Finally, one illustrative example of application is given.     

Extended Gauss–Markov Theorem for Nonparametric Mixed-Effects Models

The Gauss–Markov theorem provides a golden standard for constructing the best linear unbiased estimation for linear models. The main purpose of this article is to extend the Gauss–Markov theorem to include nonparametric mixed-effects models. The extended Gauss–Markov estimation (or prediction) is shown to be equivalent to a regularization method and its minimaxity is addressed. The resulting Gauss–Markov estimation serves as an oracle to guide the exploration for effective nonlinear estimators adaptively. Various examples are discussed. Particularly, the wavelet nonparametric regression example and its connection with a Sobolev regularization is presented.     

隐马模型及其在基因识别中的应用

生物信息学是一门新兴交叉学科,隐马模型是广泛用于该学科的数学模型.简要介绍了隐马模型的数学原理,并以大肠杆菌和人的基因识别为例说明了它在基因识别中的应用.     

Arbres de Markov couplePairwise Markov trees

The Hidden Markov Chain (HMC) models are widely applied in various problems. This succes is mainly due to the fact that the hidden model distribution conditional on observations remains a Markov chain distribution, and thus different processings, like Bayesian restorations, are handleable. These models have been recetly generalized to “Pairwise” Markov chains, which admit the same processing power and a better modeling one. The aim of this Note is to show that the Hidden Markov trees, which can be seen as extensions of the HMC models, can also be generalized to “Pairwise” Markov trees, which present the same processing advantages and better modelling power. To cite this article: W. Pieczynski, C. R. Acad. Sci. Paris, Ser. I 335 (2002) 79–82.     

On markov models of random fields

In this paper, we consider different types of Markov models for random fields, namely, causal-type (nonsymmetrical half-plane) models, causal-type (quadrant) models, semicausal-type (half-plane) models and noncausal-type models. Theorems are proved to give the spectral characterization of these types of Markov models.     

Large deviations for random dynamical systems and applications to hidden Markov models

In this paper, we prove the large deviation principle (LDP) for the occupation measures of not necessarily irreducible random dynamical systems driven by Markov processes. The LDP for not necessarily irreducible dynamical systems driven by i.i.d. sequence is derived. As a further application we establish the LDP for extended hidden Markov models, filling a gap in the literature, and obtain large deviation estimations for the log-likelihood process and maximum likelihood estimator of hidden Markov models.     

Selecting hidden Markov model state number with cross-validated likelihood

The problem of estimating the number of hidden states in a hidden Markov model is considered. Emphasis is placed on cross-validated likelihood criteria. Using cross-validation to assess the number of hidden states allows to circumvent the well-documented technical difficulties of the order identification problem in mixture models. Moreover, in a predictive perspective, it does not require that the sampling distribution belongs to one of the models in competition. However, computing cross-validated likelihood for hidden Markov models for which only one training sample is available, involves difficulties since the data are not independent. Two approaches are proposed to compute cross-validated likelihood for a hidden Markov model. The first one consists of using a deterministic half-sampling procedure, and the second one consists of an adaptation of the EM algorithm for hidden Markov models, to take into account randomly missing values induced by cross-validation. Numerical experiments on both simulated and real data sets compare different versions of cross-validated likelihood criterion and penalised likelihood criteria, including BIC and a penalised marginal likelihood criterion. Those numerical experiments highlight a promising behaviour of the deterministic half-sampling criterion.