A multivariate Markov chain model for categorical data sequences and its applications in demand predictions

Markov models are commonly used in modelling many practicalsystems such as telecommunication systems, manufacturing systemsand inventory systems. In this paper we propose a multivariateMarkov chain model for modelling multiple categorical data sequences.We develop efficient estimation methods for the model parameters.We then apply the model and method to demand predictions fora soft-drink company in Hong Kong.     

A Markov Model for the Dynamics of Cracks of a Special Type

Stochastic Markov models describe various natural and technical processes. They are often used in the most diverse fields. We single out the Markov models with discrete time and small number of states. In specific cases, such models allow carrying out effective analysis and calculations. We discuss in detail the models with four states. The processes associated with the elliptic cracks are simulated.     

Where to treat the older patient? Can Markov models help us better understand the relationship between hospital and community care?

We have previously used Markov models to describe movements of patients between hospital states; these may be actual or virtual and described by a phase-type distribution. Here we extend this approach to a Markov reward model for a healthcare system with Poisson admissions and an absorbing state, typically death. The distribution of costs is evaluated for any time and expressions derived for the mean and variances of costs. The average cost at any time is then determined for two scenarios: the Therapeutic and Prosthetic models, respectively. This example is used to illustrate the idea that keeping acute patients longer in hospital to ensure fitness for discharge, may reduce costs by decreasing the number of patients that become long-stay. In addition we develop a Markov Reward Model for a healthcare system including states, where the patient is in hospital, and states, where the patient is in the community. In each case, the length of stay is described by a phase-type distribution, thus enabling the representation of durations and costs in each phase within a Markov framework. The model can be used to determine costs for the entire system thus facilitating a systems approach to the planning of healthcare and a holistic approach to costing. Such models help us to assess the complex relationship between hospital and community care.     

Stationnarité et β-mélange des processus bilinéaires généraux à changement de régime markovien

We study the class of general bilinear models in which the parameters are allowed to depend on an unobserved Markov chain. Necessary and sufficient conditions for strict and second-order stationarity, existence of higher-order moments and conditions ensuring the geometric ergodicity are proposed.     

Assessing Convergence and Mixing of MCMC Implementations via Stratification

Some posterior distributions lead to Markov chain Monte Carlo (MCMC) chains that are naturally viewed as collections of subchains. Examples include mixture models, regime-switching models, and hidden Markov models. We obtain MCMC-based estimators of posterior expectations by combining different subgroup (subchain) estimators using stratification and poststratification methods. Variance estimates of the limiting distributions of such estimators are developed. Based on these variance estimates, we propose a test statistic to aid in the assessment of convergence and mixing of chains. We compare our diagnostic with other commonly used methods. The approach is illustrated in two examples: a latent variable model for arsenic concentration in public water systems in Arizona and a Bayesian hierarchical model for Pacific sea surface temperatures. Supplementary materials, which include MATLAB codes for the proposed method, are available online.     

Bayesian Monte Carlo estimation for profile hidden Markov models

Hidden Markov models are used as tools for pattern recognition in a number of areas, ranging from speech processing to biological sequence analysis. Profile hidden Markov models represent a class of so-called “left–right” models that have an architecture that is specifically relevant to classification of proteins into structural families based on their amino acid sequences. Standard learning methods for such models employ a variety of heuristics applied to the expectation-maximization implementation of the maximum likelihood estimation procedure in order to find the global maximum of the likelihood function. Here, we compare maximum likelihood estimation to fully Bayesian estimation of parameters for profile hidden Markov models with a small number of parameters. We find that, relative to maximum likelihood methods, Bayesian methods assign higher scores to data sequences that are distantly related to the pattern consensus, show better performance in classifying these sequences correctly, and continue to perform robustly with regard to misspecification of the number of model parameters. Though our study is limited in scope, we expect our results to remain relevant for models with a large number of parameters and other types of left–right hidden Markov models.     

A Event Occurrence Rules based Compact Modeling Formalism for a Class of Discrete Event Systems

The analysis, failure diagnosis and control of discrete event systems (DESs) requires an accurate model of the system. In this paper we present a methodology which makes the task of modeling DESs considerably less cumbersome, less error prone, and more user-friendly than it usually is. In doing so we simplify the modeling formalism of [4, 5], proposed for obtaining valid models of complex discrete event systems, by eliminating ‘precedence relations’, and capturing them as part of the ‘event occurrence rules’. Under the new modeling formalism the size of the system model is polynomial in the number of signals; whereas the number of states in the commonly used automata models is exponential in the number of signals. We present automated techniques for deriving an automaton model from the model in the proposed formalism. We illustrate the modeling formalism using examples drawn from manufacturing and process control systems.     

A hidden Markov regime-switching model for option valuation

We investigate two approaches, namely, the Esscher transform and the extended Girsanov’s principle, for option valuation in a discrete-time hidden Markov regime-switching Gaussian model. The model’s parameters including the interest rate, the appreciation rate and the volatility of a risky asset are governed by a discrete-time, finite-state, hidden Markov chain whose states represent the hidden states of an economy. We give a recursive filter for the hidden Markov chain and estimates of model parameters using a filter-based EM algorithm. We also derive predictors for the hidden Markov chain and some related quantities. These quantities are used to estimate the price of a standard European call option. Numerical examples based on real financial data are provided to illustrate the implementation of the proposed method.     

Matrix-analytic solution of infinite,finite and level-dependent second-order fluid models

This paper presents a matrix-analytic solution for second-order Markov fluid models (also known as Markov-modulated Brownian motion) with level-dependent behavior. A set of thresholds is given that divide the fluid buffer into homogeneous regimes. The generator matrix of the background Markov chain, the fluid rates (drifts) and the variances can be regime dependent. The model allows the mixing of second-order states (with positive variance) and first-order states (with zero variance) and states with zero drift. The behavior at the upper and lower boundary can be reflecting, absorbing, or a combination of them. In every regime, the solution is expressed as a matrix-exponential combination, whose matrix parameters are given by the minimal nonnegative solution of matrix quadratic equations that can be obtained by any of the well-known solution methods available for quasi birth death processes. The probability masses and the initial vectors of the matrix-exponential terms are the solutions of a set of linear equations. However, to have the necessary number of equations, new relations are required for the level boundary behavior, relations that were not needed in first-order level dependent and in homogeneous (non-level-dependent) second-order fluid models. The method presented can solve systems with hundreds of states and hundreds of thresholds without numerical issues.     

Two classes of time-inhomogeneous Markov chains: Analysis of the periodic case

We consider the M/G/1 and GI/M/1 types of Markov chains for which their one step transitions depend on the times of the transitions. These types of Markov chains are encountered in several stochastic models, including queueing systems, dams, inventory systems, insurance risk models, etc. We show that for the cases when the time parameters are periodic the systems can be analyzed using some extensions of known results in the matrix-analytic methods literature. We have limited our examples to those relating to queueing systems to allow us a focus. An example application of the model to a real life problem is presented.     

A structural model of the higher‐order Markov process incorporating reversion effects

The paper presents a formal approach which may increase the realism and parsimony of higher‐order Markov models applied to certain human behaviors. Often in behavioral applications, any improvements in fit available from increasing the order of a Markov model would be more than offset by interpretive problems caused by the very rapid increase in the number of independent parameters. The model proposed here for the higher‐order process greatly reduces the number of independent parameters, replacing them with sociologically relevant effects of persistence in and reversion to previous conditions.

The general model is called the “reversion model.” In it, individuals are allowed to carry along some information about their pasts, for a number of periods corresponding to the order of the model. The parameters describing residence histories are constructed to give each individual an underlying set of first‐order transition probabilities, which are modified by experience of the various states of the system. When an individual occupies a particular state, his relative probability of future residence there (vis‐a‐vis the other states as a group) is permitted to change. But occupation of a particular state is not permitted to affect the relative chances of residence among the other states. With suitable constraints, the number of parameters of this higher‐order process no longer increases geometrically with the order, but only arithmetically.

Maximum likelihood estimation formulas are derived for the reversion model, which is then applied to longitudinal data on the work activities of U.S. Ph.D. physicists and chemists in 1960–1966, and is found to fit well using likelihood ratio tests.     

����ɷ򽻻�ָ��Levyģ������Ȩ�۸�Ļ���-΢�ַ���

??We consider a Markov switching exponential Levy model in which the
underlying economy switches between a finite number of states. The switching is modeled by a
hidden Markov chain. We explore the link between options prices in Markov switching exponential
Levy models and the related partial integro-differential equations in the case of European
options.     

Sensitivity analysis of discrete Markov chains via matrix calculus

Markov chains are often used as mathematical models of natural phenomena, with transition probabilities defined in terms of parameters that are of interest in the scientific question at hand. Sensitivity analysis is an important way to quantify the effects of changes in these parameters on the behavior of the chain. Many properties of Markov chains can be written as simple matrix expressions, and hence matrix calculus is a powerful approach to sensitivity analysis. Using matrix calculus, we derive the sensitivity and elasticity of a variety of properties of absorbing and ergodic finite-state chains. For absorbing chains, we present the sensitivities of the moments of the number of visits to each transient state, the moments of the time to absorption, the mean number of states visited before absorption, the quasistationary distribution, and the probabilities of absorption in each of several absorbing states. For ergodic chains, we present the sensitivity of the stationary distribution, the mean first passage time matrix, the fundamental matrix, and the Kemeny constant. We include two examples of application of the results to demographic and ecological problems.     

Case Influence Analysis in Bayesian Inference

Abstract

We demonstrate how case influence analysis, commonly used in regression, can be applied to Bayesian hierarchical models. Draws from the joint posterior distribution of parameters are importance weighted to reflect the effect of deleting each observation in turn; the ensuing changes in the posterior distribution of each parameter are displayed graphically. The procedure is particularly useful when drawing a sample from the posterior distribution requires extensive calculations (as with a Markov Chain Monte Carlo sampler). The structure of hierarchical models, and other models with local dependence, makes the importance weights inexpensive to calculate with little additional programming. Some new alternative weighting schemes are described that extend the range of problems in which reweighting can be used to assess influence. Applications to a growth curve model and a complex hierarchical model for opinion data are described. Our focus on case influence on parameters is complementary to other work that measures influence by distances between posterior or predictive distributions.     

Modelling and inference with Conditional Gaussian Probabilistic Decision Graphs

Probabilistic Decision Graphs (PDGs) are probabilistic graphical models that represent a factorisation of a discrete joint probability distribution using a “decision graph”-like structure over local marginal parameters. The structure of a PDG enables the model to capture some context specific independence relations that are not representable in the structure of more commonly used graphical models such as Bayesian networks and Markov networks. This sometimes makes operations in PDGs more efficient than in alternative models. PDGs have previously been defined only in the discrete case, assuming a multinomial joint distribution over the variables in the model. We extend PDGs to incorporate continuous variables, by assuming a Conditional Gaussian (CG) joint distribution. We also show how inference can be carried out in an efficient way.     

A comparison of some criteria for states selection in the latent Markov model for longitudinal data

We compare different selection criteria to choose the number of latent states of a multivariate latent Markov model for longitudinal data. This model is based on an underlying Markov chain to represent the evolution of a latent characteristic of a group of individuals over time. Then, the response variables observed at different occasions are assumed to be conditionally independent given this chain. Maximum likelihood estimation of the model is carried out through an Expectation–Maximization algorithm based on forward–backward recursions which are well known in the hidden Markov literature for time series. The selection criteria we consider are based on penalized versions of the maximum log-likelihood or on the posterior probabilities of belonging to each latent state, that is, the conditional probability of the latent state given the observed data. Among the latter criteria, we propose an appropriate entropy measure tailored for the latent Markov models. We show the results of a Monte Carlo simulation study aimed at comparing the performance of the above states selection criteria on the basis of a wide set of model specifications.     

Selecting hidden Markov model state number with cross-validated likelihood

The problem of estimating the number of hidden states in a hidden Markov model is considered. Emphasis is placed on cross-validated likelihood criteria. Using cross-validation to assess the number of hidden states allows to circumvent the well-documented technical difficulties of the order identification problem in mixture models. Moreover, in a predictive perspective, it does not require that the sampling distribution belongs to one of the models in competition. However, computing cross-validated likelihood for hidden Markov models for which only one training sample is available, involves difficulties since the data are not independent. Two approaches are proposed to compute cross-validated likelihood for a hidden Markov model. The first one consists of using a deterministic half-sampling procedure, and the second one consists of an adaptation of the EM algorithm for hidden Markov models, to take into account randomly missing values induced by cross-validation. Numerical experiments on both simulated and real data sets compare different versions of cross-validated likelihood criterion and penalised likelihood criteria, including BIC and a penalised marginal likelihood criterion. Those numerical experiments highlight a promising behaviour of the deterministic half-sampling criterion.     

Estimation with Missing Data

This paper reviews estimation problems with missing, or hidden data. We formulate this problem in the context of Markov models and consider two interrelated issues, namely, the estimation of a state given measured data and model parameters, and the estimation of model parameters given the measured data alone. We also consider situations where the measured data is, itself, incomplete in some sense. We deal with various combinations of discrete and continuous states and observations.     

Modeling Discrete Event Systems With Faults Using a Rules-based Modeling Formalism

Obtaining accurate models of systems which are prone to failures and breakdowns is a difficult task. In this paper we present a methodology which makes the task of modeling failure prone discrete event systems (DESs) considerably less cumbersome, less error prone, and more user-friendly. The task of obtaining commonly used automata models for DESs is non-trivial for most practical systems, owing to the fact that the number of states in the commonly used automata models is exponential in the number of signals and faults. In contrast a model of a discrete event system, in the rules based modeling formalism proposed by the co-authors of this paper, is of size polynomial in the number of signals and faults. In order to model failures, we augment the signals set of the rules based formalism to include binary valued fault signals, the values representing either a non-faulty or a faulty state of a certain failure type. Addition of new fault signals requires introduction of new rules for the added fault signal events, and also modification of the existing rules for non-fault events. The rules based modeling formalism is further extended to model real-time systems, and we apply it to model delay-faults of the system as well. The model of a failure prone DES in the rules based can automatically be converted into an equivalent (timed)-automaton model for a failure analysis in the automaton model framework.     

Single-use reliability computation of a semi-Markovian system

Markov chain usage models were successfully used to model systems and software. The most prominent approaches are the so-called failure state models Whittaker and Thomason (1994) and the arc-based Bayesian models Sayre and Poore (2000). In this paper we propose arc-based semi-Markov usage models to test systems. We extend previous studies that rely on the Markov chain assumption to the more general semi-Markovian setting. Among the obtained results we give a closed form representation of the first and second moments of the single-use reliability. The model and the validity of the results are illustrated through a numerical example.