Bayesian Analysis of a Two-State Markov Modulated Poisson Process

Abstract

We postulate observations from a Poisson process whose rate parameter modulates between two values determined by an unobserved Markov chain. The theory switches from continuous to discrete time by considering the intervals between observations as a sequence of dependent random variables. A result from hidden Markov models allows us to sample from the posterior distribution of the model parameters given the observed event times using a Gibbs sampler with only two steps per iteration.     

Hidden Markov Filter Estimation of the Occurrence Time of an Event in a Financial Market

Abstract

In this paper, we use filtering techniques to estimate the occurrence time of an event in a financial market. The occurrence time is being viewed as a Markov stopping time with respect to the σ-field generated by a hidden Markov process. We also generalize our result to the Nth occurrence time of that event.     

Multi-Period Asset Allocation Under Hidden Markovianly Driven Noises

Abstract

Techniques of filtering and parameter reestimation of a general hidden Markov model are developed and applied to a discrete time multi-period asset allocation problem, where a commonly used mean-variance utility is considered and recursive calculation of an explicit optimal portfolio is provided. Our result is a generalization of that by Robert J. Elliott and John van der Hoek.     

Filtering and change point estimation for hidden Markov-modulated Poisson processes

A continuous-time Markov chain which is partially observed in Poisson noise is considered, where a structural change in the dynamics of the hidden process occurs at a random change point. Filtering and change point estimation of the model is discussed. Closed-form recursive estimates of the conditional distribution of the hidden process and the random change point are obtained, given the Poisson process observations     

Strong Markov Properties for Markov Random Fields

Markov properties and strong Markov properties for random fields are defined and discussed. Special attention is given to those defined by I. V. Evstigneev. The strong Markov nature of Markov random fields with respect to random domains such as [0, L], where L is a multidimensional extension of a stopping time, is explored. A special case of this extension is shown to generalize a result of Merzbach and Nualart for point processes. As an additional example, Evstigneev's Markov and strong Markov properties are considered for independent increment jump processes.     

Lumpability and Commutativity of Markov Processes

Abstract

We introduce the concepts of lumpability and commutativity of a continuous time discrete state space Markov process, and provide a necessary and sufficient condition for a lumpable Markov process to be commutative. Under suitable conditions we recover some of the basic quantities of the original Markov process from the jump chain of the lumped Markov process.     

Hidden Markov partition models

We describe an extension of the hidden Markov model in which the manifest process conditionally follows a partition model. The assumption of local independence for the manifest random variable is thus relaxed to arbitrary dependence. The proposed class generalizes different existing models for discrete and continuous time series, and allows for the finest trading off between bias and variance. The models are fit through an EM algorithm, with the usual recursions for hidden Markov models extended at no additional computational cost.     

The central limit theorem for stationary Markov processes with normal generator—with applications to hypergroups

We extend the central limit theorem for additive functionals of a stationary, ergodic Markov chain with normal transition operator due to Gordin and Lif?ic, 1981 [A remark about a Markov process with normal transition operator, In: Third Vilnius Conference on Probability and Statistics 1, pp. 147–48] to continuous-time Markov processes with normal generators. As examples, we discuss random walks on compact commutative hypergroups as well as certain random walks on non-commutative, compact groups.     

Estimation in Stationary Markov Renewal Processes,with Application to Earthquake Forecasting in Turkey

Consider a process in which different events occur, with random inter-occurrence times. In Markov renewal processes as well as in semi-Markov processes, the sequence of events is a Markov chain and the waiting distributions depend only on the types of the last and the next event. Suppose that the state-space is finite and that the process started far in the past, achieving stationary. Weibull distributions are proposed for the waiting times and their parameters are estimated jointly with the transition probabilities through maximum likelihood, when one or several realizations of the process are observed over finite windows. The model is illustrated with data of earthquakes of three types of severity that occurred in Turkey during the 20th century.AMS 2000 Subject Classification: 60K20     

Dempster–Shafer fusion of evidential pairwise Markov fields

Hidden Markov fields (HMFs) have been successfully used in many areas to take spatial information into account. In such models, the hidden process of interest X is a Markov field, that is to be estimated from an observable process Y. The possibility of such estimation is due to the fact that the conditional distribution of the hidden process with respect to the observed one remains Markovian. The latter property remains valid when the pairwise process $(X,Y)$ is Markov and such models, called pairwise Markov fields (PMFs), have been shown to offer larger modeling capabilities while exhibiting similar processing cost. Further extensions lead to a family of more general models called triplet Markov fields (TMFs) in which the triplet $(U,X,Y)$ is Markov where U is an underlying process that may have different meanings according to the application. A link has also been established between these models and the theory of evidence, opening new possibilities of achieving Dempster–Shafer fusion in Markov fields context. The aim of this paper is to propose a unifying general formalism allowing all conventional modeling and processing possibilities regarding information imprecision, sensor unreliability and data fusion in Markov fields context. The generality of the proposed formalism is shown theoretically through some illustrative examples dealing with image segmentation, and experimentally on hand-drawn and SAR images.     

Discrete chaotic calculus and covariance identities

We show that for the binomial process (or Bernoulli random walk) the orthogonal functionals constructed in Kroeker, J.P. (1980) "Wiener analysis of functionals of a Markov chain: application to neural transformations of random signals", Biol. Cybernetics 36 , 243-248, [14] for Markov chains can be expressed using the Krawtchouk polynomials, and by iterated stochastic integrals. This allows to construct a chaotic calculus based on gradient and divergence operators and structure equations, and to establish a Clark representation formula. As an application we obtain simple infinite dimensional proofs of covariance identities on the discrete cube.     

Separable solutions for Markov processes in random environments

In this paper we address the problem of efficiently deriving the steady-state distribution for a continuous time Markov chain (CTMC) S evolving in a random environment E. The process underlying E is also a CTMC. S is called Markov modulated process. Markov modulated processes have been widely studied in literature since they are applicable when an environment influences the behaviour of a system. For instance, this is the case of a wireless link, whose quality may depend on the state of some random factors such as the intensity of the noise in the environment. In this paper we study the class of Markov modulated processes which exhibits separable, product-form stationary distribution. We show that several models that have been proposed in literature can be studied applying the Extended Reversed Compound Agent Theorem (ERCAT), and also new product-forms are derived. We also address the problem of the necessity of ERCAT for product-forms and show a meaningful example of product-form not derivable via ERCAT.     

Bayesian Inference in Hidden Markov Random Fields for Binary Data Defined on Large Lattices

Hidden Markov random fields represent a complex hierarchical model, where the hidden latent process is an undirected graphical structure. Performing inference for such models is difficult primarily because the likelihood of the hidden states is often unavailable. The main contribution of this article is to present approximate methods to calculate the likelihood for large lattices based on exact methods for smaller lattices. We introduce approximate likelihood methods by relaxing some of the dependencies in the latent model, and also by extending tractable approximations to the likelihood, the so-called pseudolikelihood approximations, for a large lattice partitioned into smaller sublattices. Results are presented based on simulated data as well as inference for the temporal-spatial structure of the interaction between up- and down-regulated states within the mitochondrial chromosome of the Plasmodium falciparum organism. Supplemental material for this article is available online.     

Applications of Hybrid Monte Carlo to Bayesian Generalized Linear Models: Quasicomplete Separation and Neural Networks

Abstract

The “leapfrog” hybrid Monte Carlo algorithm is a simple and effective MCMC method for fitting Bayesian generalized linear models with canonical link. The algorithm leads to large trajectories over the posterior and a rapidly mixing Markov chain, having superior performance over conventional methods in difficult problems like logistic regression with quasicomplete separation. This method offers a very attractive solution to this common problem, providing a method for identifying datasets that are quasicomplete separated, and for identifying the covariates that are at the root of the problem. The method is also quite successful in fitting generalized linear models in which the link function is extended to include a feedforward neural network. With a large number of hidden units, however, or when the dataset becomes large, the computations required in calculating the gradient in each trajectory can become very demanding. In this case, it is best to mix the algorithm with multivariate random walk Metropolis—Hastings. However, this entails very little additional programming work.     

An S-Plus Implementation of Hidden Markov Models in Continuous Time

Abstract

Hidden Markov models (HMM) can be applied to the study of time varying unobserved categorical variables for which only indirect measurements are available. An S-Plus module to fit HMMs in continuous time to this type of longitudinal data is presented. Covariates affecting the transition intensities of the hidden Markov process or the conditional distribution of the measured response (given the hidden states of the process) are handled under a generalized regression framework. Users can provide C subroutines specifying the parameterization of the model to adapt the software to a wide variety of data types. HMM analysis using the S-Plus module is illustrated on a dataset from a prospective study of human papillomavirus infection in young women and on simulated data.     

非齐次隐马尔可夫模型随机变换的若干强极限定理

利用鞅方法讨论了非齐次隐马尔可夫模型变换的强极限定理,作为特殊情形,将随机选择的概念拓展到非齐次隐马尔可夫模型中,得到了关于有限非齐次隐马尔可夫模型随机选择与随机公平比的若干极限定理.     

可列非齐次隐Markov模型的强大数定律

隐马尔科夫模型被广泛的应用于弱相依随机变量的建模,是研究神经生理学、发音过程和生物遗传等问题的有力工具。研究了可列非齐次隐 Markov 模型的若干性质,得到了这类模型的强大数定律,推广了有限非齐次马氏链的一类强大数定律。     

Filtering of a Multi-Dimension Stochastic Volatility Model

In this article, we study a stochastic volatility model for a class of risky assets. We assume that the volatilities of the assets are driven by a common state of economy, which is unobservable and represented by a hidden Markov chain. Under this hidden Markov model (HMM), we develop recursively computable filtering equations for certain functionals of the chain. Expectation maximization (EM) parameter estimation is then used. Applications to an optimal asset allocation problem with mean-variance utility are given.     

Ergodic and adaptive control of hidden Markov models

A partially observed stochastic system is described by a discrete time pair of Markov processes. The observed state process has a transition probability that is controlled and depends on a hidden Markov process that also can be controlled. The hidden Markov process is completely observed in a closed set, which in particular can be the empty set and only observed through the other process in the complement of this closed set. An ergodic control problem is solved by a vanishing discount approach. In the case when the transition operators for the observed state process and the hidden Markov process depend on a parameter and the closed set, where the hidden Markov process is completely observed, is nonempty and recurrent an adaptive control is constructed based on this family of estimates that is almost optimal.